2023-07-06T21:19:46.559 INFO:root:teuthology version: 0.0.1.dev140+gb53f236 2023-07-06T21:19:46.560 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} --verbose --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242 -- /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/orig.config.yaml 2023-07-06T21:19:46.605 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T21:19:46.697 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242 branch: wip-rishabh-CephManager-in-CephFSTestCase description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} email: null first_in_suite: false job_id: '7328242' kernel: flavor: default kdb: true sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-CephManager-in-CephFSTestCase ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 0a781ef0807ae9473fb1306ef07036226ca4f1a6 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 0a781ef0807ae9473fb1306ef07036226ca4f1a6 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-CephManager-in-CephFSTestCase sha1: 0a781ef0807ae9473fb1306ef07036226ca4f1a6 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 0a781ef0807ae9473fb1306ef07036226ca4f1a6 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-CephManager-in-CephFSTestCase suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 0a781ef0807ae9473fb1306ef07036226ca4f1a6 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi084.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: b53f236c1077f1d295baafb9e0ca9a4b447019d7 timestamp: 2023-07-06_19:46:43 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-06T21:19:46.697 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa; will attempt to use it 2023-07-06T21:19:46.698 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks 2023-07-06T21:19:46.698 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-06T21:19:46.700 INFO:teuthology.task.internal:Checking packages... 2023-07-06T21:19:46.727 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '0a781ef0807ae9473fb1306ef07036226ca4f1a6' 2023-07-06T21:19:46.727 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-06T21:19:46.728 INFO:teuthology.packaging:ref: None 2023-07-06T21:19:46.728 INFO:teuthology.packaging:tag: None 2023-07-06T21:19:46.728 INFO:teuthology.packaging:branch: wip-rishabh-CephManager-in-CephFSTestCase 2023-07-06T21:19:46.728 INFO:teuthology.packaging:sha1: 0a781ef0807ae9473fb1306ef07036226ca4f1a6 2023-07-06T21:19:46.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-CephManager-in-CephFSTestCase 2023-07-06T21:19:46.942 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4488.g0a781ef0 2023-07-06T21:19:46.943 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-06T21:19:46.950 INFO:teuthology.task.internal:no buildpackages task found 2023-07-06T21:19:46.950 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-06T21:19:46.971 INFO:teuthology.task.internal:Saving configuration 2023-07-06T21:19:46.987 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-06T21:19:46.996 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T21:19:47.024 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 21:14:29.566014', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T21:19:47.047 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 21:14:29.568095', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-06T21:19:47.047 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-06T21:19:47.054 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-07-06T21:19:47.054 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-07-06T21:19:47.054 INFO:teuthology.run_tasks:Running task console_log... 2023-07-06T21:19:47.154 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4a3a544280>, signals=[15]) 2023-07-06T21:19:47.155 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-06T21:19:47.162 INFO:teuthology.task.internal:Opening connections... 2023-07-06T21:19:47.162 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:19:47.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:19:47.234 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:19:47.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:19:47.306 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-06T21:19:47.313 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2023-07-06T21:19:47.334 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2023-07-06T21:19:47.334 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2023-07-06T21:19:47.393 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T21:19:47.394 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T21:19:47.394 INFO:teuthology.orchestra.run.smithi084.stdout:ID="rhel" 2023-07-06T21:19:47.394 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="fedora" 2023-07-06T21:19:47.394 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="8.6" 2023-07-06T21:19:47.394 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el8" 2023-07-06T21:19:47.394 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:19:47.395 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T21:19:47.396 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T21:19:47.396 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T21:19:47.396 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T21:19:47.396 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2023-07-06T21:19:47.421 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2023-07-06T21:19:47.441 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2023-07-06T21:19:47.441 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2023-07-06T21:19:47.503 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="Red Hat Enterprise Linux" 2023-07-06T21:19:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="8.6 (Ootpa)" 2023-07-06T21:19:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:ID="rhel" 2023-07-06T21:19:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="fedora" 2023-07-06T21:19:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="8.6" 2023-07-06T21:19:47.504 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el8" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.redhat.com/" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-06T21:19:47.505 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:19:47.506 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-06T21:19:47.506 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-06T21:19:47.506 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-06T21:19:47.506 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-06T21:19:47.506 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2023-07-06T21:19:47.528 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-06T21:19:47.539 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-06T21:19:47.598 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-06T21:19:47.598 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2023-07-06T21:19:47.601 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2023-07-06T21:19:47.619 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-06T21:19:47.625 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-06T21:19:47.626 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2023-07-06T21:19:47.659 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2023-07-06T21:19:47.692 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-06T21:19:47.754 INFO:teuthology.run_tasks:Running task kernel... 2023-07-06T21:19:47.770 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'} 2023-07-06T21:19:47.771 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, timeout 300 2023-07-06T21:19:47.771 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T21:19:47.772 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-06T21:19:47.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:19:47.790 DEBUG:teuthology.task.kernel:sha1 for mon.b is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:19:47.791 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T21:19:47.791 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2023-07-06T21:19:47.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:19:47.792 DEBUG:teuthology.task.kernel:sha1 for mon.a is 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:19:47.792 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "84139c18d56f518615369e8310ae4ecf07a36dcd"... 2023-07-06T21:19:47.793 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2023-07-06T21:19:47.850 INFO:teuthology.orchestra.run.smithi084.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T21:19:47.851 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:19:47.851 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T21:19:47.852 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:19:47.855 INFO:teuthology.orchestra.run.smithi107.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-06T21:19:47.855 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs 84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:19:47.855 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-06T21:19:47.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:19:48.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum upgrade -y linux-firmware 2023-07-06T21:19:48.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum upgrade -y linux-firmware 2023-07-06T21:19:49.241 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-06T21:19:49.241 INFO:teuthology.orchestra.run.smithi107.stdout:Unable to read consumer identity 2023-07-06T21:19:49.309 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2023-07-06T21:19:49.310 INFO:teuthology.orchestra.run.smithi084.stdout:Unable to read consumer identity 2023-07-06T21:19:53.041 INFO:teuthology.orchestra.run.smithi107.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-06T21:19:53.353 INFO:teuthology.orchestra.run.smithi084.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-07-06T21:19:55.817 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2023-07-06T21:19:56.112 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2023-07-06T21:19:59.258 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 402 kB/s | 24 kB 00:00 2023-07-06T21:19:59.578 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 443 kB/s | 24 kB 00:00 2023-07-06T21:19:59.963 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-06T21:19:59.964 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2023-07-06T21:19:59.965 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-06T21:20:00.103 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.b... 2023-07-06T21:20:00.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:20:00.321 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2023-07-06T21:20:00.322 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2023-07-06T21:20:00.322 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2023-07-06T21:20:00.328 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T21:20:00.328 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T21:20:00.330 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi107.front.sepia.ceph.com to complete... 2023-07-06T21:20:00.461 INFO:teuthology.task.kernel:Downloading kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.a... 2023-07-06T21:20:00.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=84139c18d56f518615369e8310ae4ecf07a36dcd 2023-07-06T21:20:00.680 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ 2023-07-06T21:20:00.681 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.4.0_g84139c18d56f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-06T21:20:00.684 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi084.front.sepia.ceph.com to complete... 2023-07-06T21:20:15.505 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06 21:20:15 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T21:20:15.529 INFO:teuthology.orchestra.run.smithi107.stderr:FINISHED --2023-07-06 21:20:15-- 2023-07-06T21:20:15.530 INFO:teuthology.orchestra.run.smithi107.stderr:Total wall clock time: 15s 2023-07-06T21:20:15.530 INFO:teuthology.orchestra.run.smithi107.stderr:Downloaded: 1 files, 362M in 15s (24.3 MB/s) 2023-07-06T21:20:15.532 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.b... 2023-07-06T21:20:15.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T21:20:17.128 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying... ######################################## 2023-07-06T21:20:17.182 INFO:teuthology.orchestra.run.smithi107.stdout:Preparing... ######################################## 2023-07-06T21:20:17.278 INFO:teuthology.orchestra.run.smithi107.stdout:Updating / installing... 2023-07-06T21:20:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06 21:20:26 URL:https://2.chacra.ceph.com/r/kernel/testing/84139c18d56f518615369e8310ae4ecf07a36dcd/centos/8/flavors/default/x86_64/kernel-6.4.0_g84139c18d56f-1.x86_64.rpm [379858600/379858600] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-06T21:20:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:FINISHED --2023-07-06 21:20:26-- 2023-07-06T21:20:26.893 INFO:teuthology.orchestra.run.smithi084.stderr:Total wall clock time: 26s 2023-07-06T21:20:26.894 INFO:teuthology.orchestra.run.smithi084.stderr:Downloaded: 1 files, 362M in 26s (14.1 MB/s) 2023-07-06T21:20:26.956 INFO:teuthology.task.kernel:Installing kernel 84139c18d56f518615369e8310ae4ecf07a36dcd on mon.a... 2023-07-06T21:20:26.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-06T21:20:29.017 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying... ######################################## 2023-07-06T21:20:29.074 INFO:teuthology.orchestra.run.smithi084.stdout:Preparing... ######################################## 2023-07-06T21:20:29.157 INFO:teuthology.orchestra.run.smithi084.stdout:Updating / installing... 2023-07-06T21:20:55.364 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T21:21:06.911 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-6.4.0_g84139c18d56f-1 ######################################## 2023-07-06T21:21:21.271 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T21:21:21.271 DEBUG:teuthology.orchestra.run.smithi107:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T21:21:21.329 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T21:21:21.329 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T21:21:21.329 INFO:teuthology.orchestra.run.smithi107.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T21:21:21.330 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T21:21:21.331 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T21:21:21.332 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T21:21:21.333 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T21:21:21.334 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T21:21:21.335 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T21:21:21.336 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T21:21:21.337 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T21:21:21.338 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T21:21:21.339 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T21:21:21.340 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T21:21:21.341 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T21:21:21.342 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T21:21:21.343 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T21:21:21.344 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T21:21:21.344 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T21:21:21.344 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T21:21:21.344 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T21:21:21.344 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T21:21:21.344 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T21:21:21.345 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T21:21:21.346 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T21:21:21.347 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T21:21:21.348 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T21:21:21.349 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T21:21:21.350 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T21:21:21.351 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T21:21:21.352 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T21:21:21.353 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T21:21:21.354 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T21:21:21.355 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T21:21:21.356 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T21:21:21.357 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T21:21:21.358 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T21:21:21.359 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T21:21:21.360 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T21:21:21.361 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T21:21:21.362 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T21:21:21.363 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T21:21:21.364 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T21:21:21.365 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T21:21:21.366 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T21:21:21.367 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T21:21:21.368 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T21:21:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T21:21:21.370 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T21:21:21.371 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T21:21:21.372 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T21:21:21.373 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T21:21:21.374 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T21:21:21.375 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T21:21:21.376 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T21:21:21.377 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T21:21:21.378 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T21:21:21.379 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T21:21:21.380 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T21:21:21.381 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T21:21:21.382 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T21:21:21.383 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T21:21:21.384 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T21:21:21.385 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T21:21:21.386 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T21:21:21.387 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T21:21:21.388 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T21:21:21.389 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T21:21:21.390 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T21:21:21.391 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T21:21:21.392 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T21:21:21.393 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T21:21:21.394 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T21:21:21.395 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T21:21:21.396 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T21:21:21.397 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T21:21:21.398 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T21:21:21.399 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T21:21:21.400 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T21:21:21.401 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T21:21:21.402 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T21:21:21.403 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T21:21:21.404 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T21:21:21.405 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T21:21:21.406 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T21:21:21.407 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T21:21:21.408 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T21:21:21.409 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T21:21:21.410 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T21:21:21.411 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T21:21:21.412 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T21:21:21.413 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T21:21:21.414 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T21:21:21.415 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T21:21:21.416 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T21:21:21.417 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T21:21:21.418 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T21:21:21.419 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T21:21:21.420 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T21:21:21.421 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T21:21:21.422 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T21:21:21.423 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T21:21:21.424 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T21:21:21.425 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T21:21:21.426 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T21:21:21.427 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T21:21:21.428 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T21:21:21.429 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T21:21:21.430 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T21:21:21.431 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T21:21:21.432 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T21:21:21.433 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T21:21:21.434 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T21:21:21.435 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T21:21:21.436 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T21:21:21.437 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T21:21:21.438 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T21:21:21.439 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T21:21:21.440 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T21:21:21.441 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T21:21:21.442 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T21:21:21.443 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T21:21:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T21:21:21.445 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T21:21:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T21:21:21.447 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T21:21:21.448 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T21:21:21.449 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T21:21:21.450 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T21:21:21.451 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T21:21:21.452 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T21:21:21.453 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T21:21:21.454 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T21:21:21.455 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T21:21:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T21:21:21.457 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T21:21:21.458 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T21:21:21.459 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T21:21:21.460 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T21:21:21.461 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T21:21:21.462 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T21:21:21.463 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T21:21:21.464 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T21:21:21.465 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T21:21:21.466 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T21:21:21.467 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T21:21:21.468 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T21:21:21.469 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T21:21:21.470 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T21:21:21.471 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T21:21:21.472 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T21:21:21.473 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T21:21:21.474 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T21:21:21.475 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T21:21:21.476 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T21:21:21.477 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T21:21:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T21:21:21.479 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T21:21:21.480 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T21:21:21.481 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T21:21:21.482 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T21:21:21.483 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T21:21:21.484 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T21:21:21.485 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T21:21:21.486 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T21:21:21.487 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T21:21:21.488 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T21:21:21.489 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T21:21:21.490 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T21:21:21.491 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T21:21:21.492 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T21:21:21.493 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T21:21:21.494 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T21:21:21.495 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T21:21:21.496 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T21:21:21.497 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T21:21:21.498 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T21:21:21.499 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T21:21:21.500 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T21:21:21.501 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T21:21:21.502 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T21:21:21.503 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T21:21:21.504 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T21:21:21.505 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T21:21:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T21:21:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T21:21:21.508 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T21:21:21.509 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T21:21:21.510 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T21:21:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T21:21:21.512 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T21:21:21.513 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T21:21:21.514 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T21:21:21.515 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T21:21:21.516 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T21:21:21.517 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T21:21:21.518 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T21:21:21.519 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T21:21:21.520 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T21:21:21.521 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T21:21:21.522 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T21:21:21.523 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T21:21:21.524 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T21:21:21.525 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T21:21:21.526 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T21:21:21.527 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T21:21:21.528 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T21:21:21.529 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T21:21:21.530 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T21:21:21.531 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T21:21:21.532 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T21:21:21.533 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T21:21:21.534 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T21:21:21.535 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T21:21:21.536 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T21:21:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T21:21:21.538 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T21:21:21.539 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T21:21:21.540 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T21:21:21.541 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T21:21:21.542 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T21:21:21.543 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T21:21:21.544 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T21:21:21.545 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T21:21:21.546 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T21:21:21.547 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T21:21:21.548 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T21:21:21.549 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T21:21:21.550 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T21:21:21.551 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T21:21:21.552 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T21:21:21.553 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T21:21:21.554 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T21:21:21.555 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T21:21:21.556 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T21:21:21.557 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T21:21:21.558 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T21:21:21.559 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T21:21:21.560 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T21:21:21.561 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T21:21:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T21:21:21.563 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T21:21:21.564 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T21:21:21.565 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T21:21:21.566 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T21:21:21.567 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T21:21:21.568 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T21:21:21.569 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T21:21:21.570 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T21:21:21.571 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T21:21:21.572 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T21:21:21.573 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T21:21:21.574 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T21:21:21.575 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T21:21:21.576 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T21:21:21.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T21:21:21.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T21:21:21.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T21:21:21.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T21:21:21.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T21:21:21.577 INFO:teuthology.orchestra.run.smithi107.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T21:21:21.578 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T21:21:21.578 DEBUG:teuthology.orchestra.run.smithi107:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T21:21:21.605 INFO:teuthology.orchestra.run.smithi107.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T21:21:21.605 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T21:21:21.606 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T21:21:21.607 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T21:21:21.608 INFO:teuthology.orchestra.run.smithi107.stdout:fi 2023-07-06T21:21:21.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2023-07-06T21:21:21.688 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2023-07-06T21:21:21.688 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2023-07-06T21:21:21.688 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.02 2023-07-06T21:21:21.688 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 123.el8 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Group : System Environment/Base 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 9265450 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-06T21:21:21.689 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:Relocations : (not relocatable) 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : Red Hat, Inc. 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : Red Hat, Inc. 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2023-07-06T21:21:21.690 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:21:21.691 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T21:21:21.691 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T21:21:21.691 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T21:21:21.691 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2023-07-06T21:21:21.691 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:21:21.691 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T21:21:21.692 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T21:21:21.692 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 6.4.0-g84139c18d56f 2023-07-06T21:21:21.693 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T21:21:24.437 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2023-07-06T21:21:26.295 INFO:teuthology.orchestra.run.smithi107.stderr:done 2023-07-06T21:21:26.297 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2023-07-06T21:21:26.314 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.mvFkUS18eh 2023-07-06T21:21:26.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mvFkUS18eh 2023-07-06T21:21:26.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.mvFkUS18eh 2023-07-06T21:21:26.541 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.mvFkUS18eh is 6KB 2023-07-06T21:21:26.588 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.mvFkUS18eh 2023-07-06T21:21:26.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T21:21:26.668 INFO:teuthology.orchestra.run.smithi107.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-06T21:21:26.669 INFO:teuthology.orchestra.run.smithi107.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-06T21:21:26.669 INFO:teuthology.orchestra.run.smithi107.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.4.0-g84139c18d56f.conf 2023-07-06T21:21:26.670 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.4.0-g84139c18d56f 2023-07-06T21:21:26.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2023-07-06T21:21:33.723 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-06T21:21:33.724 DEBUG:teuthology.orchestra.run.smithi084:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-06T21:21:33.785 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/System.map-6.4.0-g84139c18d56f 2023-07-06T21:21:33.785 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/config-6.4.0-g84139c18d56f 2023-07-06T21:21:33.786 INFO:teuthology.orchestra.run.smithi084.stdout:/boot/vmlinuz-6.4.0-g84139c18d56f 2023-07-06T21:21:33.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f 2023-07-06T21:21:33.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel 2023-07-06T21:21:33.786 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch 2023-07-06T21:21:33.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86 2023-07-06T21:21:33.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto 2023-07-06T21:21:33.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-06T21:21:33.787 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-06T21:21:33.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-06T21:21:33.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-06T21:21:33.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-06T21:21:33.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-06T21:21:33.788 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-06T21:21:33.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-06T21:21:33.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-06T21:21:33.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-06T21:21:33.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-06T21:21:33.789 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-06T21:21:33.790 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-06T21:21:33.791 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/arch/x86/kvm/kvm.ko 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ansi_cprng.ko 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/anubis.ko 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/arc4.ko 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_memcpy.ko 2023-07-06T21:21:33.792 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_pq.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_tx.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/async_xor.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/async_tx/raid6test.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authenc.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/authencesn.ko 2023-07-06T21:21:33.793 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blake2b_generic.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_common.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/blowfish_generic.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/camellia_generic.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast5_generic.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast6_generic.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cast_common.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/ccm.ko 2023-07-06T21:21:33.794 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cmac.ko 2023-07-06T21:21:33.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crc32_generic.ko 2023-07-06T21:21:33.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_engine.ko 2023-07-06T21:21:33.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/crypto_user.ko 2023-07-06T21:21:33.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/cts.ko 2023-07-06T21:21:33.795 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/des_generic.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/dh_generic.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/echainiv.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/essiv.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/fcrypt.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/khazad.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4.ko 2023-07-06T21:21:33.796 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/lz4hc.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/md4.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/michael_mic.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcbc.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/pcrypt.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/rmd160.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/seed.ko 2023-07-06T21:21:33.797 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/serpent_generic.ko 2023-07-06T21:21:33.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tcrypt.ko 2023-07-06T21:21:33.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/tea.ko 2023-07-06T21:21:33.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_common.ko 2023-07-06T21:21:33.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/twofish_generic.ko 2023-07-06T21:21:33.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/vmac.ko 2023-07-06T21:21:33.798 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/wp512.ko 2023-07-06T21:21:33.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xcbc.ko 2023-07-06T21:21:33.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xor.ko 2023-07-06T21:21:33.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/crypto/xxhash_generic.ko 2023-07-06T21:21:33.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers 2023-07-06T21:21:33.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi 2023-07-06T21:21:33.799 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-06T21:21:33.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/acpi_pad.ko 2023-07-06T21:21:33.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/custom_method.ko 2023-07-06T21:21:33.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/ec_sys.ko 2023-07-06T21:21:33.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/platform_profile.ko 2023-07-06T21:21:33.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbs.ko 2023-07-06T21:21:33.800 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/sbshc.ko 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/acpi/video.ko 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/acard-ahci.ko 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ahci_platform.ko 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/ata_generic.ko 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/libahci_platform.ko 2023-07-06T21:21:33.801 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_acpi.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ali.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_amd.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_artop.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atiixp.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_atp867x.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd640.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cmd64x.ko 2023-07-06T21:21:33.802 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_cypress.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_efar.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt366.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt37x.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it8213.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_it821x.ko 2023-07-06T21:21:33.803 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_jmicron.ko 2023-07-06T21:21:33.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_marvell.ko 2023-07-06T21:21:33.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_mpiix.ko 2023-07-06T21:21:33.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_netcell.ko 2023-07-06T21:21:33.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ninja32.ko 2023-07-06T21:21:33.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87410.ko 2023-07-06T21:21:33.804 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_ns87415.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_oldpiix.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_opti.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_optidma.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pcmcia.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-06T21:21:33.805 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_piccolo.ko 2023-07-06T21:21:33.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_rdc.ko 2023-07-06T21:21:33.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sch.ko 2023-07-06T21:21:33.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_serverworks.ko 2023-07-06T21:21:33.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sil680.ko 2023-07-06T21:21:33.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sis.ko 2023-07-06T21:21:33.806 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_sl82c105.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_triflex.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pata_via.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/pdc_adma.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_inic162x.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_mv.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_nv.ko 2023-07-06T21:21:33.807 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_promise.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_qstor.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sil24.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sis.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_svw.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_sx4.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_uli.ko 2023-07-06T21:21:33.808 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_via.ko 2023-07-06T21:21:33.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ata/sata_vsc.ko 2023-07-06T21:21:33.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay 2023-07-06T21:21:33.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/auxdisplay/ks0108.ko 2023-07-06T21:21:33.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base 2023-07-06T21:21:33.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap 2023-07-06T21:21:33.809 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/bcma/bcma.ko 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/aoe/aoe.ko 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/brd.ko 2023-07-06T21:21:33.810 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/drbd/drbd.ko 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/floppy.ko 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/loop.ko 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/nbd.ko 2023-07-06T21:21:33.811 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/null_blk/null_blk.ko 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/pktcdvd.ko 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/rbd.ko 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/virtio_blk.ko 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-06T21:21:33.812 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/xen-blkfront.ko 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/block/zram/zram.ko 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hangcheck-timer.ko 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-06T21:21:33.813 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/via-rng.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-06T21:21:33.814 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/lp.ko 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/mwave/mwave.ko 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/ppdev.ko 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tlclk.ko 2023-07-06T21:21:33.815 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm 2023-07-06T21:21:33.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm.ko 2023-07-06T21:21:33.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-06T21:21:33.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-06T21:21:33.816 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/uv_mmtimer.ko 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/char/virtio_console.ko 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-06T21:21:33.817 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp 2023-07-06T21:21:33.818 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-06T21:21:33.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/ccp/ccp.ko 2023-07-06T21:21:33.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel 2023-07-06T21:21:33.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat 2023-07-06T21:21:33.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common 2023-07-06T21:21:33.819 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-06T21:21:33.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-06T21:21:33.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-06T21:21:33.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-aes.ko 2023-07-06T21:21:33.820 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/padlock-sha.ko 2023-07-06T21:21:33.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio 2023-07-06T21:21:33.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-06T21:21:33.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca 2023-07-06T21:21:33.821 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dca/dca.ko 2023-07-06T21:21:33.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq 2023-07-06T21:21:33.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-06T21:21:33.822 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma 2023-07-06T21:21:33.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw 2023-07-06T21:21:33.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-06T21:21:33.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat 2023-07-06T21:21:33.823 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-06T21:21:33.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac 2023-07-06T21:21:33.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/amd64_edac.ko 2023-07-06T21:21:33.824 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/e752x_edac.ko 2023-07-06T21:21:33.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/edac_mce_amd.ko 2023-07-06T21:21:33.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3000_edac.ko 2023-07-06T21:21:33.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i3200_edac.ko 2023-07-06T21:21:33.825 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5100_edac.ko 2023-07-06T21:21:33.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i5400_edac.ko 2023-07-06T21:21:33.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7300_edac.ko 2023-07-06T21:21:33.826 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i7core_edac.ko 2023-07-06T21:21:33.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/i82975x_edac.ko 2023-07-06T21:21:33.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/ie31200_edac.ko 2023-07-06T21:21:33.827 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/sb_edac.ko 2023-07-06T21:21:33.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/edac/x38_edac.ko 2023-07-06T21:21:33.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware 2023-07-06T21:21:33.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/edd.ko 2023-07-06T21:21:33.828 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-06T21:21:33.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio 2023-07-06T21:21:33.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-ich.ko 2023-07-06T21:21:33.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-06T21:21:33.829 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid 2023-07-06T21:21:33.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-appleir.ko 2023-07-06T21:21:33.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-aureal.ko 2023-07-06T21:21:33.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-axff.ko 2023-07-06T21:21:33.830 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-dr.ko 2023-07-06T21:21:33.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elecom.ko 2023-07-06T21:21:33.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-elo.ko 2023-07-06T21:21:33.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-emsff.ko 2023-07-06T21:21:33.831 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gaff.ko 2023-07-06T21:21:33.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gt683r.ko 2023-07-06T21:21:33.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-gyration.ko 2023-07-06T21:21:33.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-06T21:21:33.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-06T21:21:33.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-holtekff.ko 2023-07-06T21:21:33.832 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-hyperv.ko 2023-07-06T21:21:33.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-icade.ko 2023-07-06T21:21:33.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-keytouch.ko 2023-07-06T21:21:33.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-kye.ko 2023-07-06T21:21:33.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lcpower.ko 2023-07-06T21:21:33.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-led.ko 2023-07-06T21:21:33.833 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-lenovo.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-multitouch.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-ortek.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-penmount.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-petalynx.ko 2023-07-06T21:21:33.834 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-picolcd.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-pl.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-plantronics.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-primax.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-rmi.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-common.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-06T21:21:33.835 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-06T21:21:33.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-06T21:21:33.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-06T21:21:33.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-06T21:21:33.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-06T21:21:33.836 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-06T21:21:33.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-06T21:21:33.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-06T21:21:33.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-roccat.ko 2023-07-06T21:21:33.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-saitek.ko 2023-07-06T21:21:33.837 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-samsung.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sjoy.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sony.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-speedlink.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-steelseries.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-sunplus.ko 2023-07-06T21:21:33.838 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-06T21:21:33.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tivo.ko 2023-07-06T21:21:33.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-tmff.ko 2023-07-06T21:21:33.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-topseed.ko 2023-07-06T21:21:33.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-twinhan.ko 2023-07-06T21:21:33.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-uclogic.ko 2023-07-06T21:21:33.839 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-waltop.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-wiimote.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-xinmo.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zpff.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/hid-zydacron.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/uhid.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hid/wacom.ko 2023-07-06T21:21:33.840 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_balloon.ko 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_utils.ko 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hv/hv_vmbus.ko 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru.ko 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/abituguru3.ko 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-06T21:21:33.841 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7414.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ad7418.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adc128d818.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1025.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1026.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1029.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm1031.ko 2023-07-06T21:21:33.842 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adm9240.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ads7828.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7410.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7411.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7462.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7470.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7475.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/adt7x10.ko 2023-07-06T21:21:33.843 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/amc6821.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/applesmc.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asb100.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asc7621.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/atxp1.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/coretemp.ko 2023-07-06T21:21:33.844 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/dme1737.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds1621.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ds620.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc1403.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/emc6w201.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71805f.ko 2023-07-06T21:21:33.845 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f71882fg.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/f75375s.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fam15h_power.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/fschmd.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g760a.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/g762.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl518sm.ko 2023-07-06T21:21:33.846 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/gl520sm.ko 2023-07-06T21:21:33.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-06T21:21:33.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5500_temp.ko 2023-07-06T21:21:33.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/i5k_amb.ko 2023-07-06T21:21:33.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmaem.ko 2023-07-06T21:21:33.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ibmpex.ko 2023-07-06T21:21:33.847 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina209.ko 2023-07-06T21:21:33.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ina2xx.ko 2023-07-06T21:21:33.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/it87.ko 2023-07-06T21:21:33.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k10temp.ko 2023-07-06T21:21:33.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/k8temp.ko 2023-07-06T21:21:33.848 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lineage-pem.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm63.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm73.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm75.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm77.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm78.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm80.ko 2023-07-06T21:21:33.849 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm83.ko 2023-07-06T21:21:33.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm85.ko 2023-07-06T21:21:33.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm87.ko 2023-07-06T21:21:33.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm90.ko 2023-07-06T21:21:33.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm92.ko 2023-07-06T21:21:33.850 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm93.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95234.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95241.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/lm95245.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc2945.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4151.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4215.ko 2023-07-06T21:21:33.851 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4222.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4245.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4260.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/ltc4261.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max16065.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1619.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max1668.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max197.ko 2023-07-06T21:21:33.852 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6639.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6650.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/max6697.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/mcp3021.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6683.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775-core.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct6775.ko 2023-07-06T21:21:33.853 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/nct7802.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87360.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pc87427.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pcf8591.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-06T21:21:33.854 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-06T21:21:33.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-06T21:21:33.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-06T21:21:33.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-06T21:21:33.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-06T21:21:33.855 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-06T21:21:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-06T21:21:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-06T21:21:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-06T21:21:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/powr1220.ko 2023-07-06T21:21:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5627.ko 2023-07-06T21:21:33.856 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch5636.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht15.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sht21.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/shtc1.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/sis5595.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47b397.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m1.ko 2023-07-06T21:21:33.857 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/smsc47m192.ko 2023-07-06T21:21:33.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/thmc50.ko 2023-07-06T21:21:33.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp102.ko 2023-07-06T21:21:33.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp103.ko 2023-07-06T21:21:33.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp401.ko 2023-07-06T21:21:33.858 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/tmp421.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via-cputemp.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/via686a.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt1211.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/vt8231.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627ehf.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83627hf.ko 2023-07-06T21:21:33.859 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83781d.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83791d.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83792d.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83793.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83795.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l785ts.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/hwmon/w83l786ng.ko 2023-07-06T21:21:33.860 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-06T21:21:33.861 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-06T21:21:33.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-06T21:21:33.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-06T21:21:33.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-06T21:21:33.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-06T21:21:33.862 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-06T21:21:33.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-06T21:21:33.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-06T21:21:33.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-06T21:21:33.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-06T21:21:33.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-06T21:21:33.863 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-06T21:21:33.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-06T21:21:33.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-06T21:21:33.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-06T21:21:33.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-06T21:21:33.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-06T21:21:33.864 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-dev.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-mux.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-smbus.ko 2023-07-06T21:21:33.865 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/i2c/i2c-stub.ko 2023-07-06T21:21:33.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband 2023-07-06T21:21:33.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core 2023-07-06T21:21:33.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-06T21:21:33.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_core.ko 2023-07-06T21:21:33.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-06T21:21:33.866 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-06T21:21:33.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-06T21:21:33.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-06T21:21:33.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-06T21:21:33.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw 2023-07-06T21:21:33.867 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca 2023-07-06T21:21:33.868 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-06T21:21:33.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp 2023-07-06T21:21:33.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib 2023-07-06T21:21:33.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-06T21:21:33.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser 2023-07-06T21:21:33.869 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-06T21:21:33.870 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/gameport.ko 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/lightning.ko 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/gameport/ns558.ko 2023-07-06T21:21:33.871 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/joydev.ko 2023-07-06T21:21:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard 2023-07-06T21:21:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-06T21:21:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse 2023-07-06T21:21:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/appletouch.ko 2023-07-06T21:21:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/bcm5974.ko 2023-07-06T21:21:33.872 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/cyapatp.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/sermouse.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-06T21:21:33.873 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/altera_ps2.ko 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/arc_ps2.ko 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/serio/serio_raw.ko 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/input/sparse-keymap.ko 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu 2023-07-06T21:21:33.874 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-blinkm.ko 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lm3530.ko 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-lp3944.ko 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/leds-ss4200.ko 2023-07-06T21:21:33.875 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-06T21:21:33.876 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/bcache/bcache.ko 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-bio-prison.ko 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache-smq.ko 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-cache.ko 2023-07-06T21:21:33.877 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-crypt.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-delay.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-flakey.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-log-userspace.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-multipath.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-queue-length.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-raid.ko 2023-07-06T21:21:33.878 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-round-robin.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-service-time.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-switch.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-thin-pool.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/dm-verity.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/faulty.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/linear.ko 2023-07-06T21:21:33.879 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/multipath.ko 2023-07-06T21:21:33.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data 2023-07-06T21:21:33.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-06T21:21:33.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid0.ko 2023-07-06T21:21:33.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid1.ko 2023-07-06T21:21:33.880 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid10.ko 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/md/raid456.ko 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptbase.ko 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptctl.ko 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptfc.ko 2023-07-06T21:21:33.881 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptlan.ko 2023-07-06T21:21:33.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptsas.ko 2023-07-06T21:21:33.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptscsih.ko 2023-07-06T21:21:33.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/message/fusion/mptspi.ko 2023-07-06T21:21:33.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd 2023-07-06T21:21:33.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_ich.ko 2023-07-06T21:21:33.882 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/lpc_sch.ko 2023-07-06T21:21:33.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/mfd-core.ko 2023-07-06T21:21:33.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/sm501.ko 2023-07-06T21:21:33.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/viperboard.ko 2023-07-06T21:21:33.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/vx855.ko 2023-07-06T21:21:33.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/mfd/wl1273-core.ko 2023-07-06T21:21:33.883 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds9802als.ko 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/apds990x.ko 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/bh1770glc.ko 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710 2023-07-06T21:21:33.884 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/cb710/cb710.ko 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/at24.ko 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/eeprom/max6875.ko 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/enclosure.ko 2023-07-06T21:21:33.885 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/hpilo.ko 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29003.ko 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/isl29020.ko 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei 2023-07-06T21:21:33.886 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-me.ko 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei-txe.ko 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/mei/mei.ko 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-06T21:21:33.887 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-06T21:21:33.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_7xx1.ko 2023-07-06T21:21:33.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tifm_core.ko 2023-07-06T21:21:33.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/tsl2550.ko 2023-07-06T21:21:33.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_balloon.ko 2023-07-06T21:21:33.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci 2023-07-06T21:21:33.888 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/bonding/bonding.ko 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-06T21:21:33.889 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-06T21:21:33.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx 2023-07-06T21:21:33.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-06T21:21:33.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/dummy.ko 2023-07-06T21:21:33.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/eql.ko 2023-07-06T21:21:33.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet 2023-07-06T21:21:33.890 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com 2023-07-06T21:21:33.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-06T21:21:33.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-06T21:21:33.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-06T21:21:33.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-06T21:21:33.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390 2023-07-06T21:21:33.891 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere 2023-07-06T21:21:33.892 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-06T21:21:33.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon 2023-07-06T21:21:33.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-06T21:21:33.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera 2023-07-06T21:21:33.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-06T21:21:33.893 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-06T21:21:33.894 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-06T21:21:33.895 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom 2023-07-06T21:21:33.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-06T21:21:33.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-06T21:21:33.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-06T21:21:33.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-06T21:21:33.896 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-06T21:21:33.897 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-06T21:21:33.898 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-06T21:21:33.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-06T21:21:33.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-06T21:21:33.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco 2023-07-06T21:21:33.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic 2023-07-06T21:21:33.899 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-06T21:21:33.900 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-06T21:21:33.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink 2023-07-06T21:21:33.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-06T21:21:33.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-06T21:21:33.901 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/dnet.ko 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ethoc.ko 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/fealnx.ko 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel 2023-07-06T21:21:33.902 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-06T21:21:33.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000 2023-07-06T21:21:33.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-06T21:21:33.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e 2023-07-06T21:21:33.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-06T21:21:33.903 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k 2023-07-06T21:21:33.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-06T21:21:33.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e 2023-07-06T21:21:33.904 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-06T21:21:33.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf 2023-07-06T21:21:33.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-06T21:21:33.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb 2023-07-06T21:21:33.905 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-06T21:21:33.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf 2023-07-06T21:21:33.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-06T21:21:33.906 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-06T21:21:33.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-06T21:21:33.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-06T21:21:33.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-06T21:21:33.907 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/jme.ko 2023-07-06T21:21:33.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell 2023-07-06T21:21:33.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-06T21:21:33.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-06T21:21:33.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-06T21:21:33.908 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox 2023-07-06T21:21:33.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-06T21:21:33.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-06T21:21:33.909 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-06T21:21:33.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-06T21:21:33.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-06T21:21:33.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-06T21:21:33.910 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel 2023-07-06T21:21:33.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-06T21:21:33.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom 2023-07-06T21:21:33.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-06T21:21:33.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-06T21:21:33.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi 2023-07-06T21:21:33.911 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-06T21:21:33.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-06T21:21:33.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion 2023-07-06T21:21:33.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-06T21:21:33.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia 2023-07-06T21:21:33.912 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-06T21:21:33.913 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-06T21:21:33.914 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan 2023-07-06T21:21:33.915 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-06T21:21:33.916 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-06T21:21:33.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro 2023-07-06T21:21:33.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-06T21:21:33.917 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-06T21:21:33.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-06T21:21:33.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun 2023-07-06T21:21:33.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-06T21:21:33.918 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-06T21:21:33.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-06T21:21:33.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-06T21:21:33.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti 2023-07-06T21:21:33.919 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-06T21:21:33.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti 2023-07-06T21:21:33.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-06T21:21:33.920 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via 2023-07-06T21:21:33.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-06T21:21:33.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-06T21:21:33.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet 2023-07-06T21:21:33.921 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-06T21:21:33.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-06T21:21:33.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom 2023-07-06T21:21:33.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-06T21:21:33.922 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/geneve.ko 2023-07-06T21:21:33.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv 2023-07-06T21:21:33.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-06T21:21:33.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154 2023-07-06T21:21:33.923 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-06T21:21:33.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ifb.ko 2023-07-06T21:21:33.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan 2023-07-06T21:21:33.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-06T21:21:33.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvlan.ko 2023-07-06T21:21:33.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/macvtap.ko 2023-07-06T21:21:33.924 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/mii.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/net_failover.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/netconsole.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/nlmon.ko 2023-07-06T21:21:33.925 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/amd.ko 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-06T21:21:33.926 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/bcm87xx.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/broadcom.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/cicada.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/davicom.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/dp83640.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/et1011c.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/icplus.ko 2023-07-06T21:21:33.927 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/lxt.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/marvell.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/micrel.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/national.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/phylink.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/qsemi.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/realtek.ko 2023-07-06T21:21:33.928 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/smsc.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/ste10Xp.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/phy/vitesse.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_async.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-06T21:21:33.929 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppoe.ko 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pppox.ko 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/ppp/pptp.ko 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slhc.ko 2023-07-06T21:21:33.930 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/slip/slip.ko 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/sungem_phy.ko 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tap.ko 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team.ko 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-06T21:21:33.931 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_random.ko 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/tun.ko 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/veth.ko 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/virtio_net.ko 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3 2023-07-06T21:21:33.932 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/vxlan/vxlan.ko 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/net/xen-netfront.ko 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb 2023-07-06T21:21:33.933 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ntb/ntb.ko 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme-core.ko 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/nvme/host/nvme.ko 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport.ko 2023-07-06T21:21:33.934 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_cs.ko 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_pc.ko 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/parport/parport_serial.ko 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie 2023-07-06T21:21:33.935 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pci/xen-pcifront.ko 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/i82092.ko 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/pd6729.ko 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel 2023-07-06T21:21:33.936 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/acerhdf.ko 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-06T21:21:33.937 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-06T21:21:33.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-06T21:21:33.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-06T21:21:33.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-06T21:21:33.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-06T21:21:33.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-06T21:21:33.938 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-06T21:21:33.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/hdaps.ko 2023-07-06T21:21:33.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel 2023-07-06T21:21:33.939 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-06T21:21:33.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/intel_ips.ko 2023-07-06T21:21:33.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-06T21:21:33.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-06T21:21:33.940 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-06T21:21:33.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-06T21:21:33.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-06T21:21:33.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-06T21:21:33.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-06T21:21:33.941 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-06T21:21:33.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-06T21:21:33.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/platform/x86/wmi.ko 2023-07-06T21:21:33.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap 2023-07-06T21:21:33.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-06T21:21:33.942 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-06T21:21:33.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps 2023-07-06T21:21:33.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients 2023-07-06T21:21:33.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-06T21:21:33.943 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-06T21:21:33.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/clients/pps_parport.ko 2023-07-06T21:21:33.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/pps/pps_core.ko 2023-07-06T21:21:33.944 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp 2023-07-06T21:21:33.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp.ko 2023-07-06T21:21:33.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ptp/ptp_kvm.ko 2023-07-06T21:21:33.945 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc 2023-07-06T21:21:33.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-06T21:21:33.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-06T21:21:33.946 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-06T21:21:33.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-06T21:21:33.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-06T21:21:33.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-06T21:21:33.947 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-06T21:21:33.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-06T21:21:33.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-06T21:21:33.948 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-06T21:21:33.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-06T21:21:33.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-em3027.ko 2023-07-06T21:21:33.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-06T21:21:33.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-06T21:21:33.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-06T21:21:33.949 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-06T21:21:33.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-06T21:21:33.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-06T21:21:33.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-max6900.ko 2023-07-06T21:21:33.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-06T21:21:33.950 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-06T21:21:33.951 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/rtc/rtc-x1205.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-9xxx.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-sas.ko 2023-07-06T21:21:33.952 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/3w-xxxx.ko 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/BusLogic.ko 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/a100u2w.ko 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/advansys.ko 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-06T21:21:33.953 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/am53c974.ko 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/atp870u.ko 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-06T21:21:33.954 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bfa/bfa.ko 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ch.ko 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor 2023-07-06T21:21:33.955 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dc395x.ko 2023-07-06T21:21:33.956 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/dmx3191d.ko 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-06T21:21:33.957 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/esp_scsi.ko 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/fnic/fnic.ko 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hpsa.ko 2023-07-06T21:21:33.958 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hptiop.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/hv_storvsc.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/initio.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ipr.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ips.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/isci/isci.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-06T21:21:33.959 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libfc/libfc.ko 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi.ko 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/libsas/libsas.ko 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc 2023-07-06T21:21:33.960 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid.ko 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-06T21:21:33.961 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/mvumi.ko 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/pmcraid.ko 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla1280.ko 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx 2023-07-06T21:21:33.962 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/raid_class.ko 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_debug.ko 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-06T21:21:33.963 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/ses.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/st.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/stex.ko 2023-07-06T21:21:33.964 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/virtio_scsi.ko 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/scsi/wd719x.ko 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ssb/ssb.ko 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target 2023-07-06T21:21:33.965 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_file.ko 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_iblock.ko 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_mod.ko 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_pscsi.ko 2023-07-06T21:21:33.966 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/target_core_user.ko 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-06T21:21:33.967 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-06T21:21:33.968 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless 2023-07-06T21:21:33.969 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_gsm.ko 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/n_hdlc.ko 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/nozomi.ko 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-06T21:21:33.970 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/arc_uart.ko 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/tty/synclink_gt.ko 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host 2023-07-06T21:21:33.971 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio.ko 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_aec.ko 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_cif.ko 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_pci_generic.ko 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/uio/uio_sercos3.ko 2023-07-06T21:21:33.972 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-acm.ko 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usblp.ko 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/class/usbtmc.ko 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-06T21:21:33.973 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/mdc800.ko 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/image/microtek.ko 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/adutux.ko 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/appledisplay.ko 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi26.ko 2023-07-06T21:21:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/emi62.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ezusb.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/idmouse.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/iowarrior.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/ldusb.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/legousbtower.ko 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga 2023-07-06T21:21:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usb3503.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usblcd.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/uss720.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/misc/yurex.ko 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy 2023-07-06T21:21:33.976 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/phy/phy-generic.ko 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/aircable.ko 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ark3116.ko 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ch341.ko 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cp210x.ko 2023-07-06T21:21:33.977 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cyberjack.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/empeg.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/io_ti.ko 2023-07-06T21:21:33.978 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipaq.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ipw.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ir-usb.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-06T21:21:33.979 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mct_u232.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7720.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/mos7840.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/navman.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/omninet.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/opticon.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/option.ko 2023-07-06T21:21:33.980 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/oti6858.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/pl2303.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcaux.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/qcserial.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/quatech2.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/safe_serial.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/sierra.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-06T21:21:33.981 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ssu100.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/symbolserial.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_debug.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/visor.ko 2023-07-06T21:21:33.982 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/whiteheat.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/uas.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-06T21:21:33.983 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-karma.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-06T21:21:33.984 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/storage/usb-storage.ko 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci 2023-07-06T21:21:33.985 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio.ko 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost.ko 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-06T21:21:33.986 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_net.ko 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/vhost/vhost_scsi.ko 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/apple_bl.ko 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/lcd.ko 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio 2023-07-06T21:21:33.987 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_balloon.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_mmio.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/virtio/virtio_ring.ko 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1 2023-07-06T21:21:33.988 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2482.ko 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/masters/ds2490.ko 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-06T21:21:33.989 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-06T21:21:33.990 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/w1/wire.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/hpwdt.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/i6300esb.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-06T21:21:33.991 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ib700wdt.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ibmasr.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/it87_wdt.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/machzwd.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/nv_tco.ko 2023-07-06T21:21:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/softdog.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/via_wdt.ko 2023-07-06T21:21:33.993 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/wdt_pci.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/watchdog/xen_wdt.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-evtchn.ko 2023-07-06T21:21:33.994 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntalloc.ko 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-gntdev.ko 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xen-privcmd.ko 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs 2023-07-06T21:21:33.995 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/9p/9p.ko 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/affs/affs.ko 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/befs/befs.ko 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/binfmt_misc.ko 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs 2023-07-06T21:21:33.996 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/btrfs/btrfs.ko 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cachefiles/cachefiles.ko 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ceph/ceph.ko 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/coda/coda.ko 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs 2023-07-06T21:21:33.997 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/cramfs/cramfs.ko 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/dlm/dlm.ko 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/f2fs/f2fs.ko 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat 2023-07-06T21:21:33.998 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/fat.ko 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/msdos.ko 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fat/vfat.ko 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fscache/fscache.ko 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/cuse.ko 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/fuse/fuse.ko 2023-07-06T21:21:33.999 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/gfs2/gfs2.ko 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfs/hfs.ko 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/hfsplus/hfsplus.ko 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs 2023-07-06T21:21:34.000 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/isofs/isofs.ko 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/jfs/jfs.ko 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/lockd/lockd.ko 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/minix/minix.ko 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs 2023-07-06T21:21:34.001 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/netfs/netfs.ko 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-06T21:21:34.002 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfs.ko 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv3.ko 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs/nfsv4.ko 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/grace.ko 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfs_common/nfs_acl.ko 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nfsd/nfsd.ko 2023-07-06T21:21:34.003 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nilfs2/nilfs2.ko 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-celtic.ko 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-centeuro.ko 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-croatian.ko 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-cyrillic.ko 2023-07-06T21:21:34.004 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-gaelic.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-greek.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-iceland.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-inuit.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-roman.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-romanian.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/mac-turkish.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1250.ko 2023-07-06T21:21:34.005 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1251.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp1255.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp737.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp775.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp850.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp852.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp855.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp857.ko 2023-07-06T21:21:34.006 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp860.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp861.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp862.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp863.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp864.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp865.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp866.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp869.ko 2023-07-06T21:21:34.007 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp874.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp932.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp936.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp949.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_cp950.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_euc-jp.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-1.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-13.ko 2023-07-06T21:21:34.008 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-14.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-15.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-2.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-3.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-4.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-5.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-6.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-7.ko 2023-07-06T21:21:34.009 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_iso8859-9.ko 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-r.ko 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-ru.ko 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_koi8-u.ko 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/nls/nls_utf8.ko 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-06T21:21:34.010 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2.ko 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-06T21:21:34.011 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/overlayfs/overlay.ko 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/pstore/ramoops.ko 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/reiserfs/reiserfs.ko 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs 2023-07-06T21:21:34.012 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/romfs/romfs.ko 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/client/cifs.ko 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_arc4.ko 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/smb/common/cifs_md4.ko 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs 2023-07-06T21:21:34.013 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/squashfs/squashfs.ko 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/sysv/sysv.ko 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/udf/udf.ko 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/ufs/ufs.ko 2023-07-06T21:21:34.014 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/fs/xfs/xfs.ko 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/rcu/rcutorture.ko 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/torture.ko 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-06T21:21:34.015 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/asn1_encoder.ko 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc-itu-t.ko 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crc8.ko 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libarc4.ko 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/crypto/libdes.ko 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/libcrc32c.ko 2023-07-06T21:21:34.016 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lru_cache.ko 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4_compress.ko 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/lz4/lz4hc_compress.ko 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/math/cordic.ko 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/raid6/raid6_pq.ko 2023-07-06T21:21:34.017 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_bm.ko 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_fsm.ko 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/ts_kmp.ko 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/lib/zstd/zstd_compress.ko 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm 2023-07-06T21:21:34.018 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/mm/hwpoison-inject.ko 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/6lowpan.ko 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_dest.ko 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_fragment.ko 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_hop.ko 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-06T21:21:34.019 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_mobility.ko 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_routing.ko 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/6lowpan/nhc_udp.ko 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/garp.ko 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/mrp.ko 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/p8022.ko 2023-07-06T21:21:34.020 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/psnap.ko 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/802/stp.ko 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/8021q/8021q.ko 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet.ko 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_fd.ko 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_rdma.ko 2023-07-06T21:21:34.021 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/9p/9pnet_virtio.ko 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/batman-adv/batman-adv.ko 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/br_netfilter.ko 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/bridge.ko 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-06T21:21:34.022 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-06T21:21:34.023 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-06T21:21:34.024 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/ebtables.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-06T21:21:34.025 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ceph/libceph.ko 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/failover.ko 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/core/pktgen.ko 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp.ko 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_diag.ko 2023-07-06T21:21:34.026 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv4.ko 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dccp/dccp_ipv6.ko 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dns_resolver/dns_resolver.ko 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/dsa_core.ko 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_brcm.ko 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_dsa.ko 2023-07-06T21:21:34.027 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_none.ko 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/dsa/tag_trailer.ko 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154.ko 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4 2023-07-06T21:21:34.028 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ah4.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/esp4.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/fou.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/gre.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/inet_diag.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_gre.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_tunnel.ko 2023-07-06T21:21:34.029 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ip_vti.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipcomp.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/ipip.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-06T21:21:34.030 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-06T21:21:34.031 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-06T21:21:34.032 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_bic.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_dctcp.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_diag.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_highspeed.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_htcp.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_hybla.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_illinois.ko 2023-07-06T21:21:34.033 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_lp.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_scalable.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_vegas.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_veno.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_westwood.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tcp_yeah.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/tunnel4.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_diag.ko 2023-07-06T21:21:34.034 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/udp_tunnel.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ah6.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/esp6.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/fou6.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_tunnel.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-06T21:21:34.035 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ip6_vti.ko 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/ipcomp6.ko 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-06T21:21:34.036 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-06T21:21:34.037 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-06T21:21:34.038 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/sit.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/tunnel6.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key 2023-07-06T21:21:34.039 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/key/af_key.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_core.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_eth.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ip6.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_netlink.ko 2023-07-06T21:21:34.040 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/l2tp/l2tp_ppp.ko 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/llc/llc.ko 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mac802154/mac802154.ko 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/mpls/mpls_gso.ko 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter 2023-07-06T21:21:34.041 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-06T21:21:34.042 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-06T21:21:34.043 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-06T21:21:34.044 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-06T21:21:34.045 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conncount.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-06T21:21:34.046 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-06T21:21:34.047 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_log_syslog.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-06T21:21:34.048 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_irc.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_sip.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nf_tables.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-06T21:21:34.049 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_log.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_chain_nat.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_compat.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_ct.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fib.ko 2023-07-06T21:21:34.050 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_hash.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_limit.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_log.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_masq.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_nat.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_numgen.ko 2023-07-06T21:21:34.051 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_queue.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_quota.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_redir.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_reject_inet.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_socket.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/nft_tunnel.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_AUDIT.ko 2023-07-06T21:21:34.052 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_CT.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_DSCP.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HL.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_HMARK.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-06T21:21:34.053 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LED.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_LOG.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NETMAP.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFLOG.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_RATEEST.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-06T21:21:34.054 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_SECMARK.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TEE.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TPROXY.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_TRACE.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_addrtype.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_bpf.ko 2023-07-06T21:21:34.055 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cgroup.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cluster.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_comment.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connbytes.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlabel.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connlimit.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_connmark.ko 2023-07-06T21:21:34.056 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_conntrack.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_cpu.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dccp.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_devgroup.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_dscp.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ecn.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_esp.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hashlimit.ko 2023-07-06T21:21:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_helper.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_hl.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipcomp.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_iprange.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_ipvs.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_l2tp.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_length.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_limit.ko 2023-07-06T21:21:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mac.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_mark.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_multiport.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nat.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_nfacct.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_osf.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_owner.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_physdev.ko 2023-07-06T21:21:34.059 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_pkttype.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_policy.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_quota.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_rateest.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_realm.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_recent.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_sctp.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_set.ko 2023-07-06T21:21:34.060 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_socket.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_state.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_statistic.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_string.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_tcpmss.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_time.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netfilter/xt_u32.ko 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink 2023-07-06T21:21:34.061 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/netlink/netlink_diag.ko 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/nsh/nsh.ko 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/openvswitch.ko 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-geneve.ko 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-gre.ko 2023-07-06T21:21:34.062 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/openvswitch/vport-vxlan.ko 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/packet/af_packet_diag.ko 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds.ko 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_rdma.ko 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/rds/rds_tcp.ko 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched 2023-07-06T21:21:34.063 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_csum.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_gact.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_ipt.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_mirred.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_nat.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_pedit.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_police.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_simple.ko 2023-07-06T21:21:34.064 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_skbedit.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/act_vlan.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_basic.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_bpf.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_flow.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_fw.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_route.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/cls_u32.ko 2023-07-06T21:21:34.065 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_cmp.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_ipset.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_meta.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_nbyte.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_text.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/em_u32.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_choke.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_codel.ko 2023-07-06T21:21:34.066 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_drr.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_fq.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_gred.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hfsc.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_hhf.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_htb.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_ingress.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio.ko 2023-07-06T21:21:34.067 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_mqprio_lib.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_multiq.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_netem.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_pie.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_plug.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_prio.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_qfq.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_red.ko 2023-07-06T21:21:34.068 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfb.ko 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_sfq.ko 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_tbf.ko 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sched/sch_teql.ko 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp.ko 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sctp/sctp_diag.ko 2023-07-06T21:21:34.069 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/sunrpc.ko 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc 2023-07-06T21:21:34.070 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/diag.ko 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/tipc/tipc.ko 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/unix/unix_diag.ko 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock.ko 2023-07-06T21:21:34.071 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys 2023-07-06T21:21:34.072 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/security/keys/trusted-keys/trusted.ko 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/kernel/virt/lib/irqbypass.ko 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.alias.bin 2023-07-06T21:21:34.073 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.bin 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.builtin.modinfo 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.dep.bin 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.devname 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.order 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.softdep 2023-07-06T21:21:34.074 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols 2023-07-06T21:21:34.075 INFO:teuthology.orchestra.run.smithi084.stdout:/lib/modules/6.4.0-g84139c18d56f/modules.symbols.bin 2023-07-06T21:21:34.075 DEBUG:teuthology.task.kernel:get_image_version: 6.4.0-g84139c18d56f 2023-07-06T21:21:34.076 DEBUG:teuthology.orchestra.run.smithi084:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-06T21:21:34.103 INFO:teuthology.orchestra.run.smithi084.stdout:postinstall scriptlet (using /bin/sh): 2023-07-06T21:21:34.103 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.4.0-g84139c18d56f -a -r /boot/System.map-6.4.0-g84139c18d56f ]; then 2023-07-06T21:21:34.103 INFO:teuthology.orchestra.run.smithi084.stdout:cp /boot/vmlinuz-6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:cp /boot/System.map-6.4.0-g84139c18d56f /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:rm -f /boot/vmlinuz-6.4.0-g84139c18d56f /boot/System.map-6.4.0-g84139c18d56f 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/installkernel 6.4.0-g84139c18d56f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:rm -f /boot/.vmlinuz-6.4.0-g84139c18d56f-rpm /boot/.System.map-6.4.0-g84139c18d56f-rpm 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-06T21:21:34.104 INFO:teuthology.orchestra.run.smithi084.stdout:new-kernel-pkg --remove 6.4.0-g84139c18d56f --rminitrd --initrdfile=/boot/initramfs-6.4.0-g84139c18d56f.img 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-install remove 6.4.0-g84139c18d56f 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/update-bootloader --remove 6.4.0-g84139c18d56f 2023-07-06T21:21:34.105 INFO:teuthology.orchestra.run.smithi084.stdout:fi 2023-07-06T21:21:34.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2023-07-06T21:21:34.240 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2023-07-06T21:21:34.240 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2023-07-06T21:21:34.240 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.02 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 123.el8 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:Group : System Environment/Base 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 9265450 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2023-07-06T21:21:34.241 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-06T21:21:34.242 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-06T21:21:34.242 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-06T21:21:34.242 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-06T21:21:34.242 INFO:teuthology.orchestra.run.smithi084.stdout:Relocations : (not relocatable) 2023-07-06T21:21:34.242 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : Red Hat, Inc. 2023-07-06T21:21:34.242 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : Red Hat, Inc. 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-06T21:21:34.243 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-06T21:21:34.244 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2023-07-06T21:21:34.244 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:21:34.244 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2023-07-06T21:21:34.245 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-06T21:21:34.246 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 6.4.0-g84139c18d56f 2023-07-06T21:21:34.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-06T21:21:36.393 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2023-07-06T21:21:38.718 INFO:teuthology.orchestra.run.smithi084.stderr:done 2023-07-06T21:21:38.721 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2023-07-06T21:21:38.746 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.xoA8FZXI0z 2023-07-06T21:21:38.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xoA8FZXI0z 2023-07-06T21:21:38.820 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.xoA8FZXI0z 2023-07-06T21:21:38.970 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.xoA8FZXI0z is 6KB 2023-07-06T21:21:38.990 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.xoA8FZXI0z 2023-07-06T21:21:39.007 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2023-07-06T21:21:39.078 INFO:teuthology.orchestra.run.smithi084.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-06T21:21:39.079 INFO:teuthology.orchestra.run.smithi084.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-06T21:21:39.079 INFO:teuthology.orchestra.run.smithi084.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.4.0-g84139c18d56f.conf 2023-07-06T21:21:39.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.4.0-g84139c18d56f 2023-07-06T21:21:39.226 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2023-07-06T21:21:56.841 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-06T21:21:56.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-06T21:21:56.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:22:09.240 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-06T21:22:09.241 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-07-06T21:22:09.241 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:22:31.412 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2023-07-06T21:22:40.420 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-06T21:22:40.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:23:09.270 DEBUG:teuthology.orchestra.remote:timed out 2023-07-06T21:23:18.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-07-06T21:23:18.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:23:25.789 DEBUG:teuthology.orchestra.run.smithi084:> true 2023-07-06T21:23:26.172 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-07-06T21:23:26.172 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.4.0-g84139c18d56f"... 2023-07-06T21:23:26.173 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2023-07-06T21:23:26.213 INFO:teuthology.orchestra.run.smithi084.stdout:6.4.0-g84139c18d56f 2023-07-06T21:23:26.213 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T21:23:26.214 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T21:23:26.214 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-06T21:23:27.215 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-06T21:23:27.216 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-06T21:23:27.333 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2023-07-06T21:23:27.360 DEBUG:teuthology.parallel:result is None 2023-07-06T21:23:40.422 DEBUG:teuthology.orchestra.remote:timed out 2023-07-06T21:23:52.424 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-06T21:23:52.425 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:23:52.874 DEBUG:teuthology.orchestra.run.smithi107:> true 2023-07-06T21:23:53.264 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-06T21:23:53.264 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.4.0-g84139c18d56f"... 2023-07-06T21:23:53.265 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2023-07-06T21:23:53.307 INFO:teuthology.orchestra.run.smithi107.stdout:6.4.0-g84139c18d56f 2023-07-06T21:23:53.307 DEBUG:teuthology.task.kernel:current kernel version is 6.4.0-g84139c18d56f vs 6.4.0-g84139c18d56f 2023-07-06T21:23:53.307 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-06T21:23:53.307 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-06T21:23:54.308 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-06T21:23:54.309 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-06T21:23:54.445 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2023-07-06T21:23:54.473 DEBUG:teuthology.parallel:result is None 2023-07-06T21:23:54.473 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-06T21:23:54.483 INFO:teuthology.task.internal:Creating test directory... 2023-07-06T21:23:54.483 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T21:23:54.490 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-06T21:23:54.533 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-06T21:23:54.540 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-06T21:23:54.612 INFO:teuthology.task.internal:Creating archive directory... 2023-07-06T21:23:54.612 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T21:23:54.617 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-06T21:23:54.674 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-06T21:23:54.689 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-06T21:23:54.689 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T21:23:54.694 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-06T21:23:54.766 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T21:23:54.772 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T21:23:54.793 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T21:23:54.800 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-06T21:23:54.804 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-06T21:23:54.815 INFO:teuthology.task.internal:Configuring sudo... 2023-07-06T21:23:54.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T21:23:54.821 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-06T21:23:54.889 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-06T21:23:54.901 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-06T21:23:54.901 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T21:23:54.906 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-06T21:23:54.948 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T21:23:55.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T21:23:55.144 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T21:23:55.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T21:23:55.299 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:23:55.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T21:23:55.367 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T21:23:55.444 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-06T21:23:55.511 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T21:23:55.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-06T21:23:55.666 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:23:55.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-06T21:23:55.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2023-07-06T21:23:55.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2023-07-06T21:23:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T21:23:55.891 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T21:23:56.325 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-06T21:23:56.335 INFO:teuthology.task.internal:Starting timer... 2023-07-06T21:23:56.336 INFO:teuthology.run_tasks:Running task pcp... 2023-07-06T21:23:56.389 INFO:teuthology.run_tasks:Running task selinux... 2023-07-06T21:23:56.417 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-06T21:23:56.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2023-07-06T21:23:56.555 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2023-07-06T21:23:56.559 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2023-07-06T21:23:56.686 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2023-07-06T21:23:56.689 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-06T21:23:56.689 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2023-07-06T21:23:56.767 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2023-07-06T21:23:56.767 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2023-07-06T21:23:56.820 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2023-07-06T21:23:56.820 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2023-07-06T21:23:56.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T21:23:56.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:23:56.892 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T21:23:56.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:23:56.963 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-06T21:23:56.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2023-07-06T21:23:57.032 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2023-07-06T21:23:57.101 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-06T21:23:57.112 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-06T21:23:57.119 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-06T21:23:57.276 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-06T21:23:57.299 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-06T21:23:57.301 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi084.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-06T21:35:43.323 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2023-07-06T21:35:43.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-07-06T21:35:43.326 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:35:43.413 DEBUG:teuthology.orchestra.run.smithi084:> true 2023-07-06T21:35:43.521 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2023-07-06T21:35:43.522 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-06T21:35:43.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-06T21:35:43.648 DEBUG:teuthology.orchestra.run.smithi107:> true 2023-07-06T21:35:43.758 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2023-07-06T21:35:43.758 INFO:teuthology.run_tasks:Running task clock... 2023-07-06T21:35:43.770 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-06T21:35:43.770 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T21:35:43.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T21:35:43.775 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-06T21:35:43.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T21:35:43.843 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T21:35:43.869 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-06T21:35:43.876 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T21:35:43.903 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-06T21:35:43.969 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2023-07-06T21:35:43.991 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2023-07-06T21:35:43.997 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2023-07-06T21:35:44.019 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2023-07-06T21:35:44.031 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T21:35:44.052 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-06T21:35:44.065 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T21:35:44.086 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-06T21:35:44.158 INFO:teuthology.orchestra.run.smithi084.stderr:bash: ntpq: command not found 2023-07-06T21:35:44.165 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T21:35:44.166 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2023-07-06T21:35:44.166 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.166 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.166 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.166 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.172 INFO:teuthology.orchestra.run.smithi107.stderr:bash: ntpq: command not found 2023-07-06T21:35:44.179 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T21:35:44.179 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2023-07-06T21:35:44.180 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.180 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.180 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.180 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:35:44.181 INFO:teuthology.run_tasks:Running task install... 2023-07-06T21:35:44.190 DEBUG:teuthology.task.install:project ceph 2023-07-06T21:35:44.190 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6'}} 2023-07-06T21:35:44.190 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6'} 2023-07-06T21:35:44.191 INFO:teuthology.task.install:Using flavor: default 2023-07-06T21:35:44.197 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-06T21:35:44.197 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-06T21:35:44.198 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6', 'tag': None, 'wait_for_package': False} 2023-07-06T21:35:44.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0a781ef0807ae9473fb1306ef07036226ca4f1a6 2023-07-06T21:35:44.200 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6', 'tag': None, 'wait_for_package': False} 2023-07-06T21:35:44.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0a781ef0807ae9473fb1306ef07036226ca4f1a6 2023-07-06T21:35:44.339 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/ 2023-07-06T21:35:44.339 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4488.g0a781ef0 2023-07-06T21:35:44.344 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/ 2023-07-06T21:35:44.344 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4488.g0a781ef0 2023-07-06T21:35:44.447 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T21:35:44.448 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:35:44.448 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T21:35:44.515 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T21:35:44.515 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0a781ef0807ae9473fb1306ef07036226ca4f1a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T21:35:44.525 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-CephManager-in-CephFSTestCase/0a781ef0807ae9473fb1306ef07036226ca4f1a6/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-06T21:35:44.525 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:35:44.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-06T21:35:44.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-06T21:35:44.595 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0a781ef0807ae9473fb1306ef07036226ca4f1a6/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-06T21:35:44.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T21:35:44.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-06T21:35:44.717 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T21:35:44.784 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2023-07-06T21:35:44.787 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2023-07-06T21:35:44.797 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-06T21:35:44.864 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2023-07-06T21:35:44.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2023-07-06T21:35:45.232 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-06T21:35:45.310 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2023-07-06T21:35:45.518 INFO:teuthology.orchestra.run.smithi107.stdout:44 files removed 2023-07-06T21:35:45.572 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T21:35:45.595 INFO:teuthology.orchestra.run.smithi084.stdout:44 files removed 2023-07-06T21:35:45.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-06T21:35:46.012 INFO:teuthology.orchestra.run.smithi107.stdout:Updating Subscription Management repositories. 2023-07-06T21:35:46.090 INFO:teuthology.orchestra.run.smithi084.stdout:Updating Subscription Management repositories. 2023-07-06T21:35:46.788 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 308 kB/s | 81 kB 00:00 2023-07-06T21:35:46.968 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 159 kB/s | 81 kB 00:00 2023-07-06T21:35:47.106 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-07-06T21:35:47.302 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-07-06T21:35:47.465 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-07-06T21:35:47.594 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 5.5 kB/s | 1.5 kB 00:00 2023-07-06T21:35:50.032 INFO:teuthology.orchestra.run.smithi084.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-07-06T21:35:50.445 INFO:teuthology.orchestra.run.smithi107.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2023-07-06T21:35:52.626 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 6.1 MB/s | 16 MB 00:02 2023-07-06T21:35:54.145 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 16 MB 00:03 2023-07-06T21:35:56.715 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 164 kB/s | 24 kB 00:00 2023-07-06T21:35:57.750 INFO:teuthology.orchestra.run.smithi084.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 47 MB 00:00 2023-07-06T21:35:58.161 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 489 kB/s | 24 kB 00:00 2023-07-06T21:35:59.204 INFO:teuthology.orchestra.run.smithi107.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 47 MB 00:00 2023-07-06T21:36:06.617 INFO:teuthology.orchestra.run.smithi084.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-07-06T21:36:07.755 INFO:teuthology.orchestra.run.smithi107.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 53 MB 00:01 2023-07-06T21:36:18.234 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T21:36:18.235 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T21:36:18.249 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T21:36:18.250 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T21:36:18.252 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T21:36:18.253 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T21:36:18.256 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T21:36:18.259 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T21:36:18.260 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T21:36:18.399 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2023-07-06T21:36:18.412 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================================================== 2023-07-06T21:36:18.412 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2023-07-06T21:36:18.412 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================================================== 2023-07-06T21:36:18.412 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2023-07-06T21:36:18.412 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T21:36:18.412 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 6.4 k 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 5.4 M 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 848 k 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 165 k 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 1.5 M 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 128 k 2023-07-06T21:36:18.413 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 1.7 M 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 7.4 M 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 47 k 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 13 M 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 48 M 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 258 k 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 161 k 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 258 k 2023-07-06T21:36:18.414 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 18 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 31 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 771 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 139 k 2023-07-06T21:36:18.415 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 197 k 2023-07-06T21:36:18.416 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 371 k 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 364 k 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 107 k 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 94 k 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 3.4 M 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 180 k 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2023-07-06T21:36:18.417 INFO:teuthology.orchestra.run.smithi084.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T21:36:18.418 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 3.7 M 2023-07-06T21:36:18.418 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 3.6 M 2023-07-06T21:36:18.418 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2023-07-06T21:36:18.418 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T21:36:18.418 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 22 M 2023-07-06T21:36:18.418 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 23 k 2023-07-06T21:36:18.419 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 2.3 M 2023-07-06T21:36:18.419 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 240 k 2023-07-06T21:36:18.419 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 4.6 M 2023-07-06T21:36:18.419 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 18 M 2023-07-06T21:36:18.419 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 15 k 2023-07-06T21:36:18.419 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 24 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 137 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T21:36:18.420 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 473 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T21:36:18.421 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 6.7 M 2023-07-06T21:36:18.422 INFO:teuthology.orchestra.run.smithi084.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T21:36:18.422 INFO:teuthology.orchestra.run.smithi084.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T21:36:18.422 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T21:36:18.422 INFO:teuthology.orchestra.run.smithi084.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T21:36:18.422 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T21:36:18.422 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T21:36:18.423 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 45 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 119 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T21:36:18.424 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T21:36:18.425 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T21:36:18.426 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T21:36:18.427 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T21:36:18.428 INFO:teuthology.orchestra.run.smithi084.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T21:36:18.429 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T21:36:18.430 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling module streams: 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql 10 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout:=========================================================================================================================== 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout:Install 128 Packages 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 3 Packages 2023-07-06T21:36:18.431 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:36:18.432 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 205 M 2023-07-06T21:36:18.432 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2023-07-06T21:36:18.942 INFO:teuthology.orchestra.run.smithi084.stdout:(1/131): ceph-18.0.0-4488.g0a781ef0.el8.x86_64. 49 kB/s | 6.4 kB 00:00 2023-07-06T21:36:19.167 INFO:teuthology.orchestra.run.smithi084.stdout:(2/131): ceph-fuse-18.0.0-4488.g0a781ef0.el8.x8 3.7 MB/s | 848 kB 00:00 2023-07-06T21:36:19.204 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T21:36:19.205 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-06T21:36:19.220 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-06T21:36:19.221 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-06T21:36:19.222 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-06T21:36:19.224 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-06T21:36:19.226 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-06T21:36:19.229 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-06T21:36:19.231 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-06T21:36:19.242 INFO:teuthology.orchestra.run.smithi084.stdout:(3/131): ceph-immutable-object-cache-18.0.0-448 2.2 MB/s | 165 kB 00:00 2023-07-06T21:36:19.368 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2023-07-06T21:36:19.381 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-06T21:36:19.381 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2023-07-06T21:36:19.381 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-06T21:36:19.382 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2023-07-06T21:36:19.382 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-06T21:36:19.382 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-06T21:36:19.382 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 6.4 k 2023-07-06T21:36:19.382 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 5.4 M 2023-07-06T21:36:19.382 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 848 k 2023-07-06T21:36:19.383 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 165 k 2023-07-06T21:36:19.383 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 1.5 M 2023-07-06T21:36:19.383 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 128 k 2023-07-06T21:36:19.383 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 1.7 M 2023-07-06T21:36:19.383 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 7.4 M 2023-07-06T21:36:19.383 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 47 k 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 13 M 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 48 M 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 258 k 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 161 k 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 258 k 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 18 k 2023-07-06T21:36:19.384 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 31 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 771 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 139 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-06T21:36:19.385 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 197 k 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 371 k 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 364 k 2023-07-06T21:36:19.386 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 107 k 2023-07-06T21:36:19.387 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 94 k 2023-07-06T21:36:19.387 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 3.4 M 2023-07-06T21:36:19.387 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 180 k 2023-07-06T21:36:19.387 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-06T21:36:19.387 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 3.7 M 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 3.6 M 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 22 M 2023-07-06T21:36:19.388 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 23 k 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 2.3 M 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 240 k 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 4.6 M 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 18 M 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4488.g0a781ef0.el8 ceph-noarch 15 k 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 24 k 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-06T21:36:19.389 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 137 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-06T21:36:19.390 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 473 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 6.7 M 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-06T21:36:19.391 INFO:teuthology.orchestra.run.smithi107.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-06T21:36:19.392 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-06T21:36:19.393 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-06T21:36:19.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 45 k 2023-07-06T21:36:19.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:18.0.0-4488.g0a781ef0.el8 ceph 119 k 2023-07-06T21:36:19.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-06T21:36:19.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-06T21:36:19.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-06T21:36:19.394 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-06T21:36:19.395 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-06T21:36:19.396 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-06T21:36:19.397 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-06T21:36:19.398 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-06T21:36:19.399 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-06T21:36:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-06T21:36:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-06T21:36:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-06T21:36:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-06T21:36:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-06T21:36:19.400 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling module streams: 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql 10 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout:=========================================================================================================================== 2023-07-06T21:36:19.401 INFO:teuthology.orchestra.run.smithi107.stdout:Install 128 Packages 2023-07-06T21:36:19.402 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 3 Packages 2023-07-06T21:36:19.402 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:36:19.402 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 205 M 2023-07-06T21:36:19.403 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2023-07-06T21:36:19.668 INFO:teuthology.orchestra.run.smithi084.stdout:(4/131): ceph-mds-18.0.0-4488.g0a781ef0.el8.x86 5.4 MB/s | 2.3 MB 00:00 2023-07-06T21:36:19.846 INFO:teuthology.orchestra.run.smithi107.stdout:(1/131): ceph-18.0.0-4488.g0a781ef0.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-07-06T21:36:19.901 INFO:teuthology.orchestra.run.smithi084.stdout:(5/131): ceph-mgr-18.0.0-4488.g0a781ef0.el8.x86 6.6 MB/s | 1.5 MB 00:00 2023-07-06T21:36:20.360 INFO:teuthology.orchestra.run.smithi084.stdout:(6/131): ceph-mon-18.0.0-4488.g0a781ef0.el8.x86 10 MB/s | 4.6 MB 00:00 2023-07-06T21:36:20.495 INFO:teuthology.orchestra.run.smithi107.stdout:(2/131): ceph-fuse-18.0.0-4488.g0a781ef0.el8.x8 1.3 MB/s | 848 kB 00:00 2023-07-06T21:36:20.585 INFO:teuthology.orchestra.run.smithi084.stdout:(7/131): ceph-common-18.0.0-4488.g0a781ef0.el8. 12 MB/s | 22 MB 00:01 2023-07-06T21:36:20.645 INFO:teuthology.orchestra.run.smithi107.stdout:(3/131): ceph-immutable-object-cache-18.0.0-448 1.1 MB/s | 165 kB 00:00 2023-07-06T21:36:20.738 INFO:teuthology.orchestra.run.smithi107.stdout:(4/131): ceph-base-18.0.0-4488.g0a781ef0.el8.x8 4.5 MB/s | 5.4 MB 00:01 2023-07-06T21:36:21.177 INFO:teuthology.orchestra.run.smithi084.stdout:(8/131): ceph-base-18.0.0-4488.g0a781ef0.el8.x8 2.3 MB/s | 5.4 MB 00:02 2023-07-06T21:36:21.294 INFO:teuthology.orchestra.run.smithi084.stdout:(9/131): ceph-selinux-18.0.0-4488.g0a781ef0.el8 209 kB/s | 24 kB 00:00 2023-07-06T21:36:21.446 INFO:teuthology.orchestra.run.smithi107.stdout:(5/131): ceph-mgr-18.0.0-4488.g0a781ef0.el8.x86 2.2 MB/s | 1.5 MB 00:00 2023-07-06T21:36:22.069 INFO:teuthology.orchestra.run.smithi084.stdout:(10/131): ceph-radosgw-18.0.0-4488.g0a781ef0.el 9.1 MB/s | 13 MB 00:01 2023-07-06T21:36:22.136 INFO:teuthology.orchestra.run.smithi084.stdout:(11/131): cephfs-mirror-18.0.0-4488.g0a781ef0.e 3.8 MB/s | 258 kB 00:00 2023-07-06T21:36:22.178 INFO:teuthology.orchestra.run.smithi084.stdout:(12/131): libcephfs-devel-18.0.0-4488.g0a781ef0 746 kB/s | 31 kB 00:00 2023-07-06T21:36:22.179 INFO:teuthology.orchestra.run.smithi107.stdout:(6/131): ceph-mon-18.0.0-4488.g0a781ef0.el8.x86 6.3 MB/s | 4.6 MB 00:00 2023-07-06T21:36:22.229 INFO:teuthology.orchestra.run.smithi107.stdout:(7/131): ceph-mds-18.0.0-4488.g0a781ef0.el8.x86 1.4 MB/s | 2.3 MB 00:01 2023-07-06T21:36:22.353 INFO:teuthology.orchestra.run.smithi084.stdout:(13/131): ceph-osd-18.0.0-4488.g0a781ef0.el8.x8 8.9 MB/s | 18 MB 00:01 2023-07-06T21:36:22.395 INFO:teuthology.orchestra.run.smithi084.stdout:(14/131): libcephfs2-18.0.0-4488.g0a781ef0.el8. 3.5 MB/s | 771 kB 00:00 2023-07-06T21:36:22.420 INFO:teuthology.orchestra.run.smithi084.stdout:(15/131): libcephsqlite-18.0.0-4488.g0a781ef0.e 2.0 MB/s | 137 kB 00:00 2023-07-06T21:36:22.445 INFO:teuthology.orchestra.run.smithi084.stdout:(16/131): librados-devel-18.0.0-4488.g0a781ef0. 2.7 MB/s | 139 kB 00:00 2023-07-06T21:36:22.487 INFO:teuthology.orchestra.run.smithi084.stdout:(17/131): libradosstriper1-18.0.0-4488.g0a781ef 6.9 MB/s | 473 kB 00:00 2023-07-06T21:36:22.563 INFO:teuthology.orchestra.run.smithi084.stdout:(18/131): python3-ceph-argparse-18.0.0-4488.g0a 594 kB/s | 45 kB 00:00 2023-07-06T21:36:22.646 INFO:teuthology.orchestra.run.smithi084.stdout:(19/131): python3-ceph-common-18.0.0-4488.g0a78 1.4 MB/s | 119 kB 00:00 2023-07-06T21:36:22.730 INFO:teuthology.orchestra.run.smithi084.stdout:(20/131): python3-cephfs-18.0.0-4488.g0a781ef0. 2.3 MB/s | 197 kB 00:00 2023-07-06T21:36:22.822 INFO:teuthology.orchestra.run.smithi084.stdout:(21/131): python3-rados-18.0.0-4488.g0a781ef0.e 4.0 MB/s | 371 kB 00:00 2023-07-06T21:36:22.924 INFO:teuthology.orchestra.run.smithi084.stdout:(22/131): python3-rbd-18.0.0-4488.g0a781ef0.el8 3.5 MB/s | 364 kB 00:00 2023-07-06T21:36:23.007 INFO:teuthology.orchestra.run.smithi084.stdout:(23/131): python3-rgw-18.0.0-4488.g0a781ef0.el8 1.3 MB/s | 107 kB 00:00 2023-07-06T21:36:23.106 INFO:teuthology.orchestra.run.smithi084.stdout:(24/131): librgw2-18.0.0-4488.g0a781ef0.el8.x86 10 MB/s | 6.7 MB 00:00 2023-07-06T21:36:23.131 INFO:teuthology.orchestra.run.smithi084.stdout:(25/131): rbd-fuse-18.0.0-4488.g0a781ef0.el8.x8 752 kB/s | 94 kB 00:00 2023-07-06T21:36:23.207 INFO:teuthology.orchestra.run.smithi084.stdout:(26/131): rbd-nbd-18.0.0-4488.g0a781ef0.el8.x86 2.3 MB/s | 180 kB 00:00 2023-07-06T21:36:23.282 INFO:teuthology.orchestra.run.smithi084.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4488.g 308 kB/s | 23 kB 00:00 2023-07-06T21:36:23.365 INFO:teuthology.orchestra.run.smithi084.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4488.g0a781ef 1.5 MB/s | 128 kB 00:00 2023-07-06T21:36:23.424 INFO:teuthology.orchestra.run.smithi084.stdout:(29/131): rbd-mirror-18.0.0-4488.g0a781ef0.el8. 11 MB/s | 3.4 MB 00:00 2023-07-06T21:36:23.633 INFO:teuthology.orchestra.run.smithi084.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4488.g0a781 6.4 MB/s | 1.7 MB 00:00 2023-07-06T21:36:23.725 INFO:teuthology.orchestra.run.smithi084.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4488.g0a 2.6 MB/s | 240 kB 00:00 2023-07-06T21:36:23.808 INFO:teuthology.orchestra.run.smithi084.stdout:(32/131): ceph-mgr-rook-18.0.0-4488.g0a781ef0.e 571 kB/s | 47 kB 00:00 2023-07-06T21:36:23.884 INFO:teuthology.orchestra.run.smithi084.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4488.g0 194 kB/s | 15 kB 00:00 2023-07-06T21:36:23.975 INFO:teuthology.orchestra.run.smithi084.stdout:(34/131): ceph-volume-18.0.0-4488.g0a781ef0.el8 2.8 MB/s | 258 kB 00:00 2023-07-06T21:36:24.067 INFO:teuthology.orchestra.run.smithi084.stdout:(35/131): cephadm-18.0.0-4488.g0a781ef0.el8.noa 1.7 MB/s | 161 kB 00:00 2023-07-06T21:36:24.176 INFO:teuthology.orchestra.run.smithi084.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 9.9 MB/s | 7.4 MB 00:00 2023-07-06T21:36:24.202 INFO:teuthology.orchestra.run.smithi084.stdout:(37/131): cephfs-top-18.0.0-4488.g0a781ef0.el8. 134 kB/s | 18 kB 00:00 2023-07-06T21:36:24.660 INFO:teuthology.orchestra.run.smithi084.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 128 kB/s | 59 kB 00:00 2023-07-06T21:36:24.663 INFO:teuthology.orchestra.run.smithi107.stdout:(8/131): ceph-osd-18.0.0-4488.g0a781ef0.el8.x86 7.1 MB/s | 18 MB 00:02 2023-07-06T21:36:24.730 INFO:teuthology.orchestra.run.smithi107.stdout:(9/131): ceph-selinux-18.0.0-4488.g0a781ef0.el8 367 kB/s | 24 kB 00:00 2023-07-06T21:36:24.819 INFO:teuthology.orchestra.run.smithi084.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 474 kB/s | 75 kB 00:00 2023-07-06T21:36:24.844 INFO:teuthology.orchestra.run.smithi084.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 459 kB/s | 306 kB 00:00 2023-07-06T21:36:24.961 INFO:teuthology.orchestra.run.smithi084.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 374 kB/s | 44 kB 00:00 2023-07-06T21:36:25.102 INFO:teuthology.orchestra.run.smithi084.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-07-06T21:36:25.144 INFO:teuthology.orchestra.run.smithi084.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.4 MB/s | 476 kB 00:00 2023-07-06T21:36:25.220 INFO:teuthology.orchestra.run.smithi084.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2023-07-06T21:36:25.253 INFO:teuthology.orchestra.run.smithi084.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-07-06T21:36:25.370 INFO:teuthology.orchestra.run.smithi084.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.1 MB/s | 173 kB 00:00 2023-07-06T21:36:25.429 INFO:teuthology.orchestra.run.smithi084.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-07-06T21:36:25.505 INFO:teuthology.orchestra.run.smithi084.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 607 kB/s | 81 kB 00:00 2023-07-06T21:36:25.538 INFO:teuthology.orchestra.run.smithi084.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-07-06T21:36:25.614 INFO:teuthology.orchestra.run.smithi084.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 161 kB/s | 18 kB 00:00 2023-07-06T21:36:25.738 INFO:teuthology.orchestra.run.smithi107.stdout:(10/131): ceph-common-18.0.0-4488.g0a781ef0.el8 3.6 MB/s | 22 MB 00:06 2023-07-06T21:36:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 366 kB/s | 49 kB 00:00 2023-07-06T21:36:25.797 INFO:teuthology.orchestra.run.smithi084.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 4.5 MB/s | 1.2 MB 00:00 2023-07-06T21:36:25.805 INFO:teuthology.orchestra.run.smithi107.stdout:(11/131): cephfs-mirror-18.0.0-4488.g0a781ef0.e 3.8 MB/s | 258 kB 00:00 2023-07-06T21:36:25.855 INFO:teuthology.orchestra.run.smithi107.stdout:(12/131): libcephfs-devel-18.0.0-4488.g0a781ef0 621 kB/s | 31 kB 00:00 2023-07-06T21:36:25.864 INFO:teuthology.orchestra.run.smithi084.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2023-07-06T21:36:25.906 INFO:teuthology.orchestra.run.smithi084.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-07-06T21:36:26.014 INFO:teuthology.orchestra.run.smithi107.stdout:(13/131): ceph-radosgw-18.0.0-4488.g0a781ef0.el 3.6 MB/s | 13 MB 00:03 2023-07-06T21:36:26.015 INFO:teuthology.orchestra.run.smithi084.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-07-06T21:36:26.040 INFO:teuthology.orchestra.run.smithi084.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-07-06T21:36:26.064 INFO:teuthology.orchestra.run.smithi107.stdout:(14/131): libcephfs2-18.0.0-4488.g0a781ef0.el8. 3.6 MB/s | 771 kB 00:00 2023-07-06T21:36:26.089 INFO:teuthology.orchestra.run.smithi107.stdout:(15/131): libcephsqlite-18.0.0-4488.g0a781ef0.e 1.8 MB/s | 137 kB 00:00 2023-07-06T21:36:26.124 INFO:teuthology.orchestra.run.smithi084.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-07-06T21:36:26.131 INFO:teuthology.orchestra.run.smithi107.stdout:(16/131): librados-devel-18.0.0-4488.g0a781ef0. 2.0 MB/s | 139 kB 00:00 2023-07-06T21:36:26.191 INFO:teuthology.orchestra.run.smithi084.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-07-06T21:36:26.215 INFO:teuthology.orchestra.run.smithi107.stdout:(17/131): libradosstriper1-18.0.0-4488.g0a781ef 3.7 MB/s | 473 kB 00:00 2023-07-06T21:36:26.241 INFO:teuthology.orchestra.run.smithi084.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 527 kB/s | 61 kB 00:00 2023-07-06T21:36:26.265 INFO:teuthology.orchestra.run.smithi107.stdout:(18/131): python3-ceph-argparse-18.0.0-4488.g0a 895 kB/s | 45 kB 00:00 2023-07-06T21:36:26.299 INFO:teuthology.orchestra.run.smithi084.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-07-06T21:36:26.324 INFO:teuthology.orchestra.run.smithi107.stdout:(19/131): python3-ceph-common-18.0.0-4488.g0a78 2.0 MB/s | 119 kB 00:00 2023-07-06T21:36:26.350 INFO:teuthology.orchestra.run.smithi084.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-07-06T21:36:26.416 INFO:teuthology.orchestra.run.smithi107.stdout:(20/131): python3-cephfs-18.0.0-4488.g0a781ef0. 2.1 MB/s | 197 kB 00:00 2023-07-06T21:36:26.425 INFO:teuthology.orchestra.run.smithi084.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-07-06T21:36:26.476 INFO:teuthology.orchestra.run.smithi084.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-07-06T21:36:26.524 INFO:teuthology.orchestra.run.smithi107.stdout:(21/131): python3-rados-18.0.0-4488.g0a781ef0.e 3.4 MB/s | 371 kB 00:00 2023-07-06T21:36:26.617 INFO:teuthology.orchestra.run.smithi084.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 1.3 MB/s | 255 kB 00:00 2023-07-06T21:36:26.624 INFO:teuthology.orchestra.run.smithi107.stdout:(22/131): python3-rbd-18.0.0-4488.g0a781ef0.el8 3.6 MB/s | 364 kB 00:00 2023-07-06T21:36:26.675 INFO:teuthology.orchestra.run.smithi107.stdout:(23/131): python3-rgw-18.0.0-4488.g0a781ef0.el8 2.1 MB/s | 107 kB 00:00 2023-07-06T21:36:26.725 INFO:teuthology.orchestra.run.smithi107.stdout:(24/131): rbd-fuse-18.0.0-4488.g0a781ef0.el8.x8 1.8 MB/s | 94 kB 00:00 2023-07-06T21:36:26.751 INFO:teuthology.orchestra.run.smithi084.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 459 kB/s | 61 kB 00:00 2023-07-06T21:36:26.785 INFO:teuthology.orchestra.run.smithi084.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 813 kB/s | 251 kB 00:00 2023-07-06T21:36:26.885 INFO:teuthology.orchestra.run.smithi084.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 639 kB/s | 85 kB 00:00 2023-07-06T21:36:26.911 INFO:teuthology.orchestra.run.smithi084.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2023-07-06T21:36:26.935 INFO:teuthology.orchestra.run.smithi084.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-07-06T21:36:27.050 INFO:teuthology.orchestra.run.smithi107.stdout:(25/131): librgw2-18.0.0-4488.g0a781ef0.el8.x86 7.3 MB/s | 6.7 MB 00:00 2023-07-06T21:36:27.117 INFO:teuthology.orchestra.run.smithi107.stdout:(26/131): rbd-nbd-18.0.0-4488.g0a781ef0.el8.x86 2.6 MB/s | 180 kB 00:00 2023-07-06T21:36:27.167 INFO:teuthology.orchestra.run.smithi107.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4488.g 464 kB/s | 23 kB 00:00 2023-07-06T21:36:27.209 INFO:teuthology.orchestra.run.smithi107.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4488.g0a781ef 3.0 MB/s | 128 kB 00:00 2023-07-06T21:36:27.303 INFO:teuthology.orchestra.run.smithi084.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 37 MB/s | 14 MB 00:00 2023-07-06T21:36:27.310 INFO:teuthology.orchestra.run.smithi107.stdout:(29/131): rbd-mirror-18.0.0-4488.g0a781ef0.el8. 5.9 MB/s | 3.4 MB 00:00 2023-07-06T21:36:27.353 INFO:teuthology.orchestra.run.smithi084.stdout:(71/131): thrift-0.13.0-2.el8.x86_64.rpm 3.7 MB/s | 1.7 MB 00:00 2023-07-06T21:36:27.378 INFO:teuthology.orchestra.run.smithi084.stdout:(72/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-07-06T21:36:27.428 INFO:teuthology.orchestra.run.smithi084.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 721 kB/s | 54 kB 00:00 2023-07-06T21:36:27.454 INFO:teuthology.orchestra.run.smithi084.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.6 MB/s | 259 kB 00:00 2023-07-06T21:36:27.477 INFO:teuthology.orchestra.run.smithi107.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4488.g0a781 6.4 MB/s | 1.7 MB 00:00 2023-07-06T21:36:27.487 INFO:teuthology.orchestra.run.smithi084.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-06T21:36:27.513 INFO:teuthology.orchestra.run.smithi084.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 667 kB/s | 39 kB 00:00 2023-07-06T21:36:27.546 INFO:teuthology.orchestra.run.smithi084.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-06T21:36:27.560 INFO:teuthology.orchestra.run.smithi107.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4488.g0a 2.8 MB/s | 240 kB 00:00 2023-07-06T21:36:27.580 INFO:teuthology.orchestra.run.smithi084.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-07-06T21:36:27.605 INFO:teuthology.orchestra.run.smithi084.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-07-06T21:36:27.610 INFO:teuthology.orchestra.run.smithi107.stdout:(32/131): ceph-mgr-rook-18.0.0-4488.g0a781ef0.e 950 kB/s | 47 kB 00:00 2023-07-06T21:36:27.639 INFO:teuthology.orchestra.run.smithi084.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2023-07-06T21:36:27.652 INFO:teuthology.orchestra.run.smithi107.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4488.g0 350 kB/s | 15 kB 00:00 2023-07-06T21:36:27.672 INFO:teuthology.orchestra.run.smithi084.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-07-06T21:36:27.706 INFO:teuthology.orchestra.run.smithi084.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 800 kB/s | 27 kB 00:00 2023-07-06T21:36:27.744 INFO:teuthology.orchestra.run.smithi107.stdout:(34/131): ceph-volume-18.0.0-4488.g0a781ef0.el8 2.8 MB/s | 258 kB 00:00 2023-07-06T21:36:27.748 INFO:teuthology.orchestra.run.smithi084.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-06T21:36:27.790 INFO:teuthology.orchestra.run.smithi084.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-07-06T21:36:27.803 INFO:teuthology.orchestra.run.smithi107.stdout:(35/131): cephadm-18.0.0-4488.g0a781ef0.el8.noa 2.7 MB/s | 161 kB 00:00 2023-07-06T21:36:27.862 INFO:teuthology.orchestra.run.smithi107.stdout:(36/131): cephfs-top-18.0.0-4488.g0a781ef0.el8. 310 kB/s | 18 kB 00:00 2023-07-06T21:36:27.865 INFO:teuthology.orchestra.run.smithi084.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-07-06T21:36:27.916 INFO:teuthology.orchestra.run.smithi084.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-06T21:36:27.958 INFO:teuthology.orchestra.run.smithi084.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-06T21:36:28.025 INFO:teuthology.orchestra.run.smithi084.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2023-07-06T21:36:28.241 INFO:teuthology.orchestra.run.smithi084.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 226 kB/s | 64 kB 00:00 2023-07-06T21:36:28.245 INFO:teuthology.orchestra.run.smithi107.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 799 kB/s | 306 kB 00:00 2023-07-06T21:36:28.275 INFO:teuthology.orchestra.run.smithi084.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 504 kB/s | 126 kB 00:00 2023-07-06T21:36:28.292 INFO:teuthology.orchestra.run.smithi084.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 396 kB/s | 20 kB 00:00 2023-07-06T21:36:28.318 INFO:teuthology.orchestra.run.smithi084.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-07-06T21:36:28.360 INFO:teuthology.orchestra.run.smithi084.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-06T21:36:28.396 INFO:teuthology.orchestra.run.smithi107.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 6.8 MB/s | 7.4 MB 00:01 2023-07-06T21:36:28.401 INFO:teuthology.orchestra.run.smithi084.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-06T21:36:28.421 INFO:teuthology.orchestra.run.smithi107.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 336 kB/s | 59 kB 00:00 2023-07-06T21:36:28.535 INFO:teuthology.orchestra.run.smithi084.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-07-06T21:36:28.563 INFO:teuthology.orchestra.run.smithi107.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.3 MB/s | 476 kB 00:00 2023-07-06T21:36:28.577 INFO:teuthology.orchestra.run.smithi084.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-06T21:36:28.652 INFO:teuthology.orchestra.run.smithi084.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-07-06T21:36:28.655 INFO:teuthology.orchestra.run.smithi107.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 477 kB/s | 44 kB 00:00 2023-07-06T21:36:28.680 INFO:teuthology.orchestra.run.smithi107.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 264 kB/s | 75 kB 00:00 2023-07-06T21:36:28.694 INFO:teuthology.orchestra.run.smithi084.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-07-06T21:36:28.747 INFO:teuthology.orchestra.run.smithi107.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-07-06T21:36:28.753 INFO:teuthology.orchestra.run.smithi084.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-07-06T21:36:28.772 INFO:teuthology.orchestra.run.smithi107.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 359 kB/s | 33 kB 00:00 2023-07-06T21:36:28.806 INFO:teuthology.orchestra.run.smithi107.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 279 kB/s | 16 kB 00:00 2023-07-06T21:36:28.836 INFO:teuthology.orchestra.run.smithi084.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-07-06T21:36:28.912 INFO:teuthology.orchestra.run.smithi084.stdout:(101/131): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-07-06T21:36:28.937 INFO:teuthology.orchestra.run.smithi084.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 467 kB/s | 47 kB 00:00 2023-07-06T21:36:28.948 INFO:teuthology.orchestra.run.smithi107.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2023-07-06T21:36:28.963 INFO:teuthology.orchestra.run.smithi084.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-06T21:36:28.988 INFO:teuthology.orchestra.run.smithi084.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-07-06T21:36:29.021 INFO:teuthology.orchestra.run.smithi084.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-07-06T21:36:29.040 INFO:teuthology.orchestra.run.smithi107.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 645 kB/s | 173 kB 00:00 2023-07-06T21:36:29.047 INFO:teuthology.orchestra.run.smithi084.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 825 kB/s | 48 kB 00:00 2023-07-06T21:36:29.065 INFO:teuthology.orchestra.run.smithi107.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 692 kB/s | 81 kB 00:00 2023-07-06T21:36:29.072 INFO:teuthology.orchestra.run.smithi084.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-06T21:36:29.097 INFO:teuthology.orchestra.run.smithi084.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-07-06T21:36:29.123 INFO:teuthology.orchestra.run.smithi084.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-06T21:36:29.124 INFO:teuthology.orchestra.run.smithi107.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-07-06T21:36:29.148 INFO:teuthology.orchestra.run.smithi084.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-06T21:36:29.149 INFO:teuthology.orchestra.run.smithi107.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 211 kB/s | 18 kB 00:00 2023-07-06T21:36:29.181 INFO:teuthology.orchestra.run.smithi084.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-06T21:36:29.199 INFO:teuthology.orchestra.run.smithi107.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 976 kB/s | 49 kB 00:00 2023-07-06T21:36:29.206 INFO:teuthology.orchestra.run.smithi084.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 320 kB/s | 19 kB 00:00 2023-07-06T21:36:29.232 INFO:teuthology.orchestra.run.smithi084.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-07-06T21:36:29.250 INFO:teuthology.orchestra.run.smithi107.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-07-06T21:36:29.265 INFO:teuthology.orchestra.run.smithi084.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.3 MB/s | 559 kB 00:00 2023-07-06T21:36:29.291 INFO:teuthology.orchestra.run.smithi084.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-07-06T21:36:29.308 INFO:teuthology.orchestra.run.smithi107.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2023-07-06T21:36:29.324 INFO:teuthology.orchestra.run.smithi084.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-06T21:36:29.350 INFO:teuthology.orchestra.run.smithi084.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-06T21:36:29.392 INFO:teuthology.orchestra.run.smithi084.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-07-06T21:36:29.392 INFO:teuthology.orchestra.run.smithi107.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-07-06T21:36:29.434 INFO:teuthology.orchestra.run.smithi107.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 376 kB/s | 16 kB 00:00 2023-07-06T21:36:29.459 INFO:teuthology.orchestra.run.smithi084.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 17 MB/s | 2.3 MB 00:00 2023-07-06T21:36:29.468 INFO:teuthology.orchestra.run.smithi107.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-07-06T21:36:29.492 INFO:teuthology.orchestra.run.smithi084.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 824 kB/s | 83 kB 00:00 2023-07-06T21:36:29.517 INFO:teuthology.orchestra.run.smithi084.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.0 MB/s | 60 kB 00:00 2023-07-06T21:36:29.518 INFO:teuthology.orchestra.run.smithi107.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-07-06T21:36:29.543 INFO:teuthology.orchestra.run.smithi084.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-06T21:36:29.560 INFO:teuthology.orchestra.run.smithi107.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 1.5 MB/s | 61 kB 00:00 2023-07-06T21:36:29.568 INFO:teuthology.orchestra.run.smithi084.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-06T21:36:29.593 INFO:teuthology.orchestra.run.smithi084.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-07-06T21:36:29.594 INFO:teuthology.orchestra.run.smithi107.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 560 kB/s | 19 kB 00:00 2023-07-06T21:36:29.627 INFO:teuthology.orchestra.run.smithi084.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-06T21:36:29.627 INFO:teuthology.orchestra.run.smithi107.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 722 kB/s | 24 kB 00:00 2023-07-06T21:36:29.652 INFO:teuthology.orchestra.run.smithi084.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-06T21:36:29.661 INFO:teuthology.orchestra.run.smithi107.stdout:(61/131): python3-kubernetes-11.0.0-6.el8.noarc 2.2 MB/s | 1.2 MB 00:00 2023-07-06T21:36:29.677 INFO:teuthology.orchestra.run.smithi084.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-06T21:36:29.678 INFO:teuthology.orchestra.run.smithi107.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-07-06T21:36:29.703 INFO:teuthology.orchestra.run.smithi107.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 663 kB/s | 27 kB 00:00 2023-07-06T21:36:29.728 INFO:teuthology.orchestra.run.smithi107.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-07-06T21:36:29.770 INFO:teuthology.orchestra.run.smithi107.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 1.4 MB/s | 61 kB 00:00 2023-07-06T21:36:29.821 INFO:teuthology.orchestra.run.smithi107.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.1 MB/s | 251 kB 00:00 2023-07-06T21:36:29.846 INFO:teuthology.orchestra.run.smithi107.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-07-06T21:36:29.871 INFO:teuthology.orchestra.run.smithi107.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 459 kB/s | 23 kB 00:00 2023-07-06T21:36:29.896 INFO:teuthology.orchestra.run.smithi107.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-06T21:36:30.080 INFO:teuthology.orchestra.run.smithi107.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 7.4 MB/s | 1.7 MB 00:00 2023-07-06T21:36:30.147 INFO:teuthology.orchestra.run.smithi107.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.6 MB/s | 110 kB 00:00 2023-07-06T21:36:30.153 INFO:teuthology.orchestra.run.smithi084.stdout:(128/131): librbd1-18.0.0-4488.g0a781ef0.el8.x8 7.5 MB/s | 3.6 MB 00:00 2023-07-06T21:36:30.197 INFO:teuthology.orchestra.run.smithi107.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-07-06T21:36:30.211 INFO:teuthology.orchestra.run.smithi084.stdout:(129/131): librados2-18.0.0-4488.g0a781ef0.el8. 6.5 MB/s | 3.7 MB 00:00 2023-07-06T21:36:30.253 INFO:teuthology.orchestra.run.smithi084.stdout:(130/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 7.8 MB/s | 798 kB 00:00 2023-07-06T21:36:30.364 INFO:teuthology.orchestra.run.smithi107.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-07-06T21:36:30.798 INFO:teuthology.orchestra.run.smithi107.stdout:(74/131): ceph-test-18.0.0-4488.g0a781ef0.el8.x 7.9 MB/s | 48 MB 00:06 2023-07-06T21:36:30.840 INFO:teuthology.orchestra.run.smithi107.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 403 kB/s | 259 kB 00:00 2023-07-06T21:36:30.873 INFO:teuthology.orchestra.run.smithi107.stdout:(76/131): socat-1.7.4.1-1.el8.x86_64.rpm 635 kB/s | 323 kB 00:00 2023-07-06T21:36:30.899 INFO:teuthology.orchestra.run.smithi107.stdout:(77/131): python3-markupsafe-0.23-19.el8.x86_64 388 kB/s | 39 kB 00:00 2023-07-06T21:36:30.941 INFO:teuthology.orchestra.run.smithi107.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.8 MB/s | 538 kB 00:00 2023-07-06T21:36:30.974 INFO:teuthology.orchestra.run.smithi107.stdout:(79/131): bison-3.0.4-10.el8.x86_64.rpm 5.2 MB/s | 688 kB 00:00 2023-07-06T21:36:31.000 INFO:teuthology.orchestra.run.smithi107.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2023-07-06T21:36:31.025 INFO:teuthology.orchestra.run.smithi107.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 2.2 MB/s | 187 kB 00:00 2023-07-06T21:36:31.075 INFO:teuthology.orchestra.run.smithi107.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 6.9 MB/s | 713 kB 00:00 2023-07-06T21:36:31.100 INFO:teuthology.orchestra.run.smithi107.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 354 kB/s | 27 kB 00:00 2023-07-06T21:36:31.134 INFO:teuthology.orchestra.run.smithi107.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 5.3 MB/s | 320 kB 00:00 2023-07-06T21:36:31.160 INFO:teuthology.orchestra.run.smithi107.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 752 kB/s | 44 kB 00:00 2023-07-06T21:36:31.209 INFO:teuthology.orchestra.run.smithi107.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-07-06T21:36:31.260 INFO:teuthology.orchestra.run.smithi107.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-07-06T21:36:31.318 INFO:teuthology.orchestra.run.smithi107.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2023-07-06T21:36:31.385 INFO:teuthology.orchestra.run.smithi107.stdout:(89/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-07-06T21:36:31.452 INFO:teuthology.orchestra.run.smithi107.stdout:(90/131): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2023-07-06T21:36:31.477 INFO:teuthology.orchestra.run.smithi107.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 218 kB/s | 20 kB 00:00 2023-07-06T21:36:31.503 INFO:teuthology.orchestra.run.smithi107.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 687 kB/s | 126 kB 00:00 2023-07-06T21:36:31.528 INFO:teuthology.orchestra.run.smithi107.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-07-06T21:36:31.570 INFO:teuthology.orchestra.run.smithi107.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-07-06T21:36:31.595 INFO:teuthology.orchestra.run.smithi107.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-07-06T21:36:31.654 INFO:teuthology.orchestra.run.smithi107.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-07-06T21:36:31.704 INFO:teuthology.orchestra.run.smithi107.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-07-06T21:36:31.796 INFO:teuthology.orchestra.run.smithi107.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 17 MB/s | 4.9 MB 00:00 2023-07-06T21:36:31.871 INFO:teuthology.orchestra.run.smithi107.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-07-06T21:36:31.922 INFO:teuthology.orchestra.run.smithi107.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.3 MB/s | 808 kB 00:00 2023-07-06T21:36:31.955 INFO:teuthology.orchestra.run.smithi107.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-07-06T21:36:31.997 INFO:teuthology.orchestra.run.smithi107.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-07-06T21:36:32.022 INFO:teuthology.orchestra.run.smithi107.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-07-06T21:36:32.048 INFO:teuthology.orchestra.run.smithi107.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-07-06T21:36:32.073 INFO:teuthology.orchestra.run.smithi107.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-06T21:36:32.098 INFO:teuthology.orchestra.run.smithi107.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 957 kB/s | 48 kB 00:00 2023-07-06T21:36:32.123 INFO:teuthology.orchestra.run.smithi107.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-07-06T21:36:32.149 INFO:teuthology.orchestra.run.smithi107.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-07-06T21:36:32.174 INFO:teuthology.orchestra.run.smithi107.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-07-06T21:36:32.208 INFO:teuthology.orchestra.run.smithi107.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2023-07-06T21:36:32.283 INFO:teuthology.orchestra.run.smithi107.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.7 MB/s | 5.1 MB 00:00 2023-07-06T21:36:32.308 INFO:teuthology.orchestra.run.smithi107.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 187 kB/s | 19 kB 00:00 2023-07-06T21:36:32.333 INFO:teuthology.orchestra.run.smithi107.stdout:(113/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.4 MB/s | 560 kB 00:00 2023-07-06T21:36:32.359 INFO:teuthology.orchestra.run.smithi107.stdout:(114/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-07-06T21:36:32.384 INFO:teuthology.orchestra.run.smithi107.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 357 kB/s | 18 kB 00:00 2023-07-06T21:36:32.418 INFO:teuthology.orchestra.run.smithi107.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 5.0 MB/s | 559 kB 00:00 2023-07-06T21:36:32.451 INFO:teuthology.orchestra.run.smithi107.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2023-07-06T21:36:32.476 INFO:teuthology.orchestra.run.smithi107.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-07-06T21:36:32.502 INFO:teuthology.orchestra.run.smithi107.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-07-06T21:36:32.527 INFO:teuthology.orchestra.run.smithi107.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-07-06T21:36:32.552 INFO:teuthology.orchestra.run.smithi107.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-07-06T21:36:32.577 INFO:teuthology.orchestra.run.smithi107.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-06T21:36:32.603 INFO:teuthology.orchestra.run.smithi107.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-06T21:36:32.628 INFO:teuthology.orchestra.run.smithi107.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-07-06T21:36:32.670 INFO:teuthology.orchestra.run.smithi107.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-07-06T21:36:32.695 INFO:teuthology.orchestra.run.smithi107.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-07-06T21:36:32.721 INFO:teuthology.orchestra.run.smithi107.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-07-06T21:36:32.771 INFO:teuthology.orchestra.run.smithi107.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.6 MB/s | 2.3 MB 00:00 2023-07-06T21:36:32.838 INFO:teuthology.orchestra.run.smithi107.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 12 MB/s | 798 kB 00:00 2023-07-06T21:36:33.377 INFO:teuthology.orchestra.run.smithi107.stdout:(130/131): librbd1-18.0.0-4488.g0a781ef0.el8.x8 5.4 MB/s | 3.6 MB 00:00 2023-07-06T21:36:33.438 INFO:teuthology.orchestra.run.smithi107.stdout:(131/131): librados2-18.0.0-4488.g0a781ef0.el8. 4.9 MB/s | 3.7 MB 00:00 2023-07-06T21:36:33.441 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2023-07-06T21:36:33.441 INFO:teuthology.orchestra.run.smithi107.stdout:Total 15 MB/s | 205 MB 00:14 2023-07-06T21:36:34.620 INFO:teuthology.orchestra.run.smithi084.stdout:(131/131): ceph-test-18.0.0-4488.g0a781ef0.el8. 3.6 MB/s | 48 MB 00:13 2023-07-06T21:36:34.625 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2023-07-06T21:36:34.625 INFO:teuthology.orchestra.run.smithi084.stdout:Total 13 MB/s | 205 MB 00:16 2023-07-06T21:36:35.050 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2023-07-06T21:36:35.182 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2023-07-06T21:36:35.183 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2023-07-06T21:36:36.255 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2023-07-06T21:36:36.388 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2023-07-06T21:36:36.388 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2023-07-06T21:36:36.902 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2023-07-06T21:36:36.907 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2023-07-06T21:36:38.109 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2023-07-06T21:36:38.113 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2023-07-06T21:36:42.014 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2023-07-06T21:36:42.201 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T21:36:42.397 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T21:36:42.611 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-06T21:36:42.793 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T21:36:42.945 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T21:36:43.143 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T21:36:43.178 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2023-07-06T21:36:43.371 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-06T21:36:43.594 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T21:36:43.775 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-06T21:36:43.860 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 6/134 2023-07-06T21:36:43.984 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-06T21:36:44.164 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T21:36:44.316 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 6/134 2023-07-06T21:36:44.348 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-06T21:36:44.526 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 7/134 2023-07-06T21:36:44.564 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T21:36:44.679 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 7/134 2023-07-06T21:36:44.884 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 8/134 2023-07-06T21:36:45.007 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 8/134 2023-07-06T21:36:45.033 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-06T21:36:45.207 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:18.0.0-4488.g0a781ef0.el8.x86_64 9/134 2023-07-06T21:36:45.272 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 6/134 2023-07-06T21:36:45.475 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T21:36:45.689 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T21:36:45.720 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 6/134 2023-07-06T21:36:45.913 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 7/134 2023-07-06T21:36:46.059 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 7/134 2023-07-06T21:36:46.134 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T21:36:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 8/134 2023-07-06T21:36:46.356 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T21:36:46.364 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 8/134 2023-07-06T21:36:46.542 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 14/134 2023-07-06T21:36:46.602 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:18.0.0-4488.g0a781ef0.el8.x86_64 9/134 2023-07-06T21:36:46.672 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 14/134 2023-07-06T21:36:46.837 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-06T21:36:46.850 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86 15/134 2023-07-06T21:36:46.950 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86 15/134 2023-07-06T21:36:47.060 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-06T21:36:47.092 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T21:36:47.258 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T21:36:47.409 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T21:36:47.531 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-06T21:36:47.781 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-06T21:36:47.955 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 14/134 2023-07-06T21:36:48.079 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 14/134 2023-07-06T21:36:48.098 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T21:36:48.263 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86 15/134 2023-07-06T21:36:48.267 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T21:36:48.364 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86 15/134 2023-07-06T21:36:48.496 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T21:36:48.889 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-06T21:36:48.949 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T21:36:49.038 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T21:36:49.150 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T21:36:49.739 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-06T21:36:49.904 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T21:36:49.984 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T21:36:50.249 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-06T21:36:50.494 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-06T21:36:50.586 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-06T21:36:50.732 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-06T21:36:50.771 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T21:36:50.901 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-06T21:36:51.109 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-06T21:36:51.378 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T21:36:51.541 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:18.0.0-4488.g0a781ef0.el8. 26/134 2023-07-06T21:36:51.589 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-06T21:36:51.688 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:18.0.0-4488.g0a781ef0.el 27/134 2023-07-06T21:36:51.879 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-06T21:36:52.121 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-06T21:36:52.369 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-06T21:36:52.546 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-06T21:36:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:18.0.0-4488.g0a781ef0.el8.x86_6 28/134 2023-07-06T21:36:52.764 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-06T21:36:53.023 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-06T21:36:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-06T21:36:53.194 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:18.0.0-4488.g0a781ef0.el8. 26/134 2023-07-06T21:36:53.308 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:18.0.0-4488.g0a781ef0.el 27/134 2023-07-06T21:36:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-06T21:36:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-06T21:36:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T21:36:54.333 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:18.0.0-4488.g0a781ef0.el8.x86_6 28/134 2023-07-06T21:36:54.691 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-06T21:36:54.977 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-06T21:36:55.276 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-06T21:36:55.470 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T21:36:55.931 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T21:36:56.325 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-06T21:36:56.544 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T21:36:56.755 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T21:36:56.974 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 36/134 2023-07-06T21:36:57.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:18.0.0-4488.g0a781ef0.el8.x86_6 37/134 2023-07-06T21:36:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-06T21:36:57.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-06T21:36:57.530 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-06T21:36:57.685 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-06T21:36:57.853 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-06T21:36:57.853 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-06T21:36:58.060 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-06T21:36:58.064 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-06T21:36:58.250 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-06T21:36:58.293 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-06T21:36:58.453 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 36/134 2023-07-06T21:36:58.633 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:18.0.0-4488.g0a781ef0.el8.x86_6 37/134 2023-07-06T21:36:58.679 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-06T21:36:58.814 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-06T21:36:59.020 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-06T21:36:59.213 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-06T21:36:59.290 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-06T21:36:59.374 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-06T21:36:59.498 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-06T21:36:59.571 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-06T21:36:59.737 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-06T21:36:59.780 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-06T21:36:59.966 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-06T21:37:00.157 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-06T21:37:00.309 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-06T21:37:00.443 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-06T21:37:00.566 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-06T21:37:00.659 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-06T21:37:00.751 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-06T21:37:00.921 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-06T21:37:00.968 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-06T21:37:01.138 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-06T21:37:01.187 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-06T21:37:01.311 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-06T21:37:01.527 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-06T21:37:01.651 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-06T21:37:01.778 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-06T21:37:01.919 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-06T21:37:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-06T21:37:02.180 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-06T21:37:02.187 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-06T21:37:02.341 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-06T21:37:02.456 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-06T21:37:02.506 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-06T21:37:02.640 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-06T21:37:02.801 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-06T21:37:02.847 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-06T21:37:03.005 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T21:37:03.099 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T21:37:03.146 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-06T21:37:03.353 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-06T21:37:03.425 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-06T21:37:03.578 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-06T21:37:03.659 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-06T21:37:03.755 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T21:37:03.835 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-06T21:37:03.844 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T21:37:03.980 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-06T21:37:04.083 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-06T21:37:04.168 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T21:37:04.247 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-06T21:37:04.280 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-06T21:37:04.499 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T21:37:04.504 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-06T21:37:04.705 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T21:37:04.719 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-06T21:37:04.876 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T21:37:04.982 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T21:37:04.986 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-06T21:37:05.196 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T21:37:05.211 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-06T21:37:05.352 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T21:37:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-06T21:37:05.526 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T21:37:05.577 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-06T21:37:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T21:37:05.775 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-06T21:37:05.921 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T21:37:06.033 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-06T21:37:06.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T21:37:06.242 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-06T21:37:06.267 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T21:37:06.397 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-06T21:37:06.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T21:37:06.538 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-06T21:37:06.625 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T21:37:06.739 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-06T21:37:06.818 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T21:37:06.933 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-06T21:37:06.994 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T21:37:07.114 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-06T21:37:07.272 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T21:37:07.287 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-06T21:37:07.424 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-06T21:37:07.530 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T21:37:07.629 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-06T21:37:07.714 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T21:37:07.840 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-06T21:37:07.935 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T21:37:08.032 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-06T21:37:08.299 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-06T21:37:08.551 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-06T21:37:08.728 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-06T21:37:08.962 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-06T21:37:09.252 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T21:37:09.525 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T21:37:09.768 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T21:37:10.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T21:37:10.297 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-06T21:37:10.391 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T21:37:10.560 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-06T21:37:10.575 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T21:37:10.838 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-06T21:37:11.160 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-06T21:37:11.410 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T21:37:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-06T21:37:11.603 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 92/134 2023-07-06T21:37:11.631 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-06T21:37:11.711 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 92/134 2023-07-06T21:37:11.879 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 93/134 2023-07-06T21:37:12.465 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-06T21:37:12.674 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 92/134 2023-07-06T21:37:12.767 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 92/134 2023-07-06T21:37:12.941 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 93/134 2023-07-06T21:37:16.670 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 94/134 2023-07-06T21:37:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 94/134 2023-07-06T21:37:17.592 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 94/134 2023-07-06T21:37:17.845 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 94/134 2023-07-06T21:37:17.896 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 95/134 2023-07-06T21:37:18.032 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 95/134 2023-07-06T21:37:18.041 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 94/134 2023-07-06T21:37:18.152 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 96/134 2023-07-06T21:37:18.717 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 94/134 2023-07-06T21:37:19.044 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 95/134 2023-07-06T21:37:19.184 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 95/134 2023-07-06T21:37:20.347 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 96/134 2023-07-06T21:37:38.235 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 96/134 2023-07-06T21:37:38.236 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2023-07-06T21:37:38.236 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2023-07-06T21:37:38.236 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /dev 2023-07-06T21:37:38.236 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /run 2023-07-06T21:37:38.236 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2023-07-06T21:37:38.237 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2023-07-06T21:37:38.237 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2023-07-06T21:37:38.237 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2023-07-06T21:37:38.237 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /dev 2023-07-06T21:37:38.237 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:37:38.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 97/134 2023-07-06T21:37:40.345 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 97/134 2023-07-06T21:37:40.507 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 98/134 2023-07-06T21:37:40.542 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 96/134 2023-07-06T21:37:40.542 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /dev 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /run 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2023-07-06T21:37:40.543 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2023-07-06T21:37:40.544 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /dev 2023-07-06T21:37:40.544 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:37:40.775 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 98/134 2023-07-06T21:37:40.931 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 99/134 2023-07-06T21:37:40.944 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 97/134 2023-07-06T21:37:41.474 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 99/134 2023-07-06T21:37:41.664 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 100/134 2023-07-06T21:37:41.704 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 100/134 2023-07-06T21:37:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 97/134 2023-07-06T21:37:42.863 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 98/134 2023-07-06T21:37:43.132 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 98/134 2023-07-06T21:37:43.293 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 99/134 2023-07-06T21:37:43.835 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 99/134 2023-07-06T21:37:44.018 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 100/134 2023-07-06T21:37:44.059 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 100/134 2023-07-06T21:37:44.301 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 101/134 2023-07-06T21:37:44.477 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 101/134 2023-07-06T21:37:44.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4488.g0a781ef0.e 102/134 2023-07-06T21:37:44.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4488.g0a781ef0. 103/134 2023-07-06T21:37:45.030 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noa 104/134 2023-07-06T21:37:45.295 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noa 104/134 2023-07-06T21:37:45.489 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.n 105/134 2023-07-06T21:37:46.601 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.n 105/134 2023-07-06T21:37:46.727 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 101/134 2023-07-06T21:37:46.788 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a7 106/134 2023-07-06T21:37:46.907 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 101/134 2023-07-06T21:37:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a7 106/134 2023-07-06T21:37:47.067 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4488.g0a781ef0.e 102/134 2023-07-06T21:37:47.288 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4488.g0a781ef0. 103/134 2023-07-06T21:37:47.297 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4488.g0a781ef0.el 107/134 2023-07-06T21:37:47.473 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 108/134 2023-07-06T21:37:47.483 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noa 104/134 2023-07-06T21:37:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 108/134 2023-07-06T21:37:47.744 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noa 104/134 2023-07-06T21:37:47.745 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 109/134 2023-07-06T21:37:47.774 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 109/134 2023-07-06T21:37:47.953 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.n 105/134 2023-07-06T21:37:49.060 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.n 105/134 2023-07-06T21:37:49.177 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:18.0.0-4488.g0a781ef0.el8.x86_64 110/134 2023-07-06T21:37:49.241 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a7 106/134 2023-07-06T21:37:49.369 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a7 106/134 2023-07-06T21:37:49.422 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 111/134 2023-07-06T21:37:49.484 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 111/134 2023-07-06T21:37:49.629 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4488.g0a781 112/134 2023-07-06T21:37:49.700 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4488.g0a781 112/134 2023-07-06T21:37:49.778 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4488.g0a781ef0.el 107/134 2023-07-06T21:37:49.854 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 113/134 2023-07-06T21:37:49.954 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 108/134 2023-07-06T21:37:50.018 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 108/134 2023-07-06T21:37:50.182 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 109/134 2023-07-06T21:37:50.210 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 109/134 2023-07-06T21:37:50.264 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 113/134 2023-07-06T21:37:50.421 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 114/134 2023-07-06T21:37:51.611 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:18.0.0-4488.g0a781ef0.el8.x86_64 110/134 2023-07-06T21:37:51.858 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 111/134 2023-07-06T21:37:51.920 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 111/134 2023-07-06T21:37:52.074 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4488.g0a781 112/134 2023-07-06T21:37:52.145 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4488.g0a781 112/134 2023-07-06T21:37:52.308 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 113/134 2023-07-06T21:37:52.714 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 113/134 2023-07-06T21:37:52.866 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 114/134 2023-07-06T21:37:55.391 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 114/134 2023-07-06T21:37:55.698 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:18.0.0-4488.g0a781ef0.el8.x86_64 115/134 2023-07-06T21:37:55.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T21:37:56.096 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T21:37:57.358 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-06T21:37:57.715 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-06T21:37:57.860 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 114/134 2023-07-06T21:37:57.905 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 119/134 2023-07-06T21:37:58.132 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T21:37:58.161 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:18.0.0-4488.g0a781ef0.el8.x86_64 115/134 2023-07-06T21:37:58.314 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T21:37:58.315 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:18.0.0-4488.g0a781ef0.el8.x86_ 121/134 2023-07-06T21:37:58.461 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T21:37:58.561 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-06T21:37:59.773 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-06T21:38:00.135 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-06T21:38:00.316 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 119/134 2023-07-06T21:38:00.536 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T21:38:00.710 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:18.0.0-4488.g0a781ef0.el8.x86_ 121/134 2023-07-06T21:38:00.865 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T21:38:01.734 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T21:38:01.993 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T21:38:02.032 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T21:38:02.269 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:18.0.0-4488.g0a781ef0.el8.noarch 124/134 2023-07-06T21:38:02.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 125/134 2023-07-06T21:38:02.824 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 126/134 2023-07-06T21:38:03.144 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T21:38:03.325 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T21:38:03.348 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T21:38:03.559 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-06T21:38:03.759 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T21:38:03.867 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T21:38:04.092 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T21:38:04.120 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T21:38:04.121 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T21:38:04.154 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T21:38:04.357 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T21:38:04.357 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T21:38:04.463 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T21:38:04.502 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-06T21:38:04.632 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T21:38:04.714 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:18.0.0-4488.g0a781ef0.el8.noarch 124/134 2023-07-06T21:38:04.834 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T21:38:04.922 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 125/134 2023-07-06T21:38:05.103 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T21:38:05.104 INFO:teuthology.orchestra.run.smithi107.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T21:38:05.104 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:05.239 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 126/134 2023-07-06T21:38:05.565 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-06T21:38:05.702 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:18.0.0-4488.g0a781ef0.el8.x86_64 1/134 2023-07-06T21:38:05.702 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 2/134 2023-07-06T21:38:05.702 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 3/134 2023-07-06T21:38:05.702 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 4/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4488.g0a781 5/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 6/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 7/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 8/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 9/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 10/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 11/134 2023-07-06T21:38:05.703 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:18.0.0-4488.g0a781ef0.el8.x86_64 12/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 13/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:18.0.0-4488.g0a781ef0.el8.x86_ 14/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 15/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 16/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:18.0.0-4488.g0a781ef0.el8.x86_6 17/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86 18/134 2023-07-06T21:38:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 19/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4488.g0a781ef0.el 20/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:18.0.0-4488.g0a781ef0.el8. 21/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:18.0.0-4488.g0a781ef0.el8.x86_6 22/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:18.0.0-4488.g0a781ef0.el8.x86_64 23/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 24/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 25/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 26/134 2023-07-06T21:38:05.705 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 27/134 2023-07-06T21:38:05.706 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 28/134 2023-07-06T21:38:05.712 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4488.g0a781ef0. 29/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noa 30/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.n 31/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a7 32/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4488.g0a781ef0.el 33/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 34/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4488.g0a781ef0.e 35/134 2023-07-06T21:38:05.713 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 36/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 37/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:18.0.0-4488.g0a781ef0.el8.noarch 38/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T21:38:05.714 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T21:38:05.715 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T21:38:05.716 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T21:38:05.717 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-06T21:38:05.718 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-06T21:38:05.719 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-06T21:38:05.720 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-06T21:38:05.721 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-06T21:38:05.722 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-06T21:38:05.723 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-06T21:38:05.724 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 129/134 2023-07-06T21:38:05.725 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T21:38:05.726 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 131/134 2023-07-06T21:38:05.726 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T21:38:05.726 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T21:38:05.737 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T21:38:05.761 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T21:38:05.987 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-06T21:38:06.170 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T21:38:06.279 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-06T21:38:06.521 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T21:38:06.549 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-06T21:38:06.549 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T21:38:06.768 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T21:38:06.769 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T21:38:07.011 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-06T21:38:07.212 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T21:38:07.484 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T21:38:07.484 INFO:teuthology.orchestra.run.smithi084.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-06T21:38:07.485 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:08.122 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:18.0.0-4488.g0a781ef0.el8.x86_64 1/134 2023-07-06T21:38:08.122 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 2/134 2023-07-06T21:38:08.122 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 3/134 2023-07-06T21:38:08.122 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 4/134 2023-07-06T21:38:08.122 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4488.g0a781 5/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 6/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 7/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 8/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 9/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 10/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 11/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:18.0.0-4488.g0a781ef0.el8.x86_64 12/134 2023-07-06T21:38:08.123 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 13/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:18.0.0-4488.g0a781ef0.el8.x86_ 14/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 15/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 16/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:18.0.0-4488.g0a781ef0.el8.x86_6 17/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86 18/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 19/134 2023-07-06T21:38:08.124 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4488.g0a781ef0.el 20/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:18.0.0-4488.g0a781ef0.el8. 21/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:18.0.0-4488.g0a781ef0.el8.x86_6 22/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:18.0.0-4488.g0a781ef0.el8.x86_64 23/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 24/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 25/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 26/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 27/134 2023-07-06T21:38:08.125 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 28/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4488.g0a781ef0. 29/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noa 30/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.n 31/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a7 32/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4488.g0a781ef0.el 33/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 34/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4488.g0a781ef0.e 35/134 2023-07-06T21:38:08.126 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 36/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 37/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:18.0.0-4488.g0a781ef0.el8.noarch 38/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-06T21:38:08.127 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-06T21:38:08.128 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-06T21:38:08.129 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-06T21:38:08.130 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-06T21:38:08.131 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-06T21:38:08.132 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-06T21:38:08.133 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-06T21:38:08.134 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-06T21:38:08.135 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-06T21:38:08.136 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-06T21:38:08.137 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-06T21:38:08.138 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 129/134 2023-07-06T21:38:08.139 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-06T21:38:08.139 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 131/134 2023-07-06T21:38:08.139 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-06T21:38:08.139 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-06T21:38:08.612 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T21:38:08.612 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:02:01 ago on Thu 06 Jul 2023 09:36:07 PM UTC. 2023-07-06T21:38:10.832 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-06T21:38:10.833 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:02:04 ago on Thu 06 Jul 2023 09:36:06 PM UTC. 2023-07-06T21:38:11.232 INFO:teuthology.orchestra.run.smithi107.stdout:Installed products updated. 2023-07-06T21:38:11.499 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:11.500 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2023-07-06T21:38:11.500 INFO:teuthology.orchestra.run.smithi107.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T21:38:11.500 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.500 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.500 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.501 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.502 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.503 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.504 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.504 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.504 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:11.504 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T21:38:11.504 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T21:38:11.504 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: jq-1.6-3.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.505 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T21:38:11.506 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T21:38:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T21:38:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T21:38:11.507 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T21:38:11.508 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T21:38:11.509 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.510 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T21:38:11.511 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T21:38:11.512 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T21:38:11.513 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T21:38:11.514 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T21:38:11.515 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T21:38:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T21:38:11.517 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T21:38:11.518 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T21:38:11.518 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T21:38:11.518 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T21:38:11.518 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T21:38:11.518 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:11.519 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2023-07-06T21:38:11.853 DEBUG:teuthology.parallel:result is None 2023-07-06T21:38:13.463 INFO:teuthology.orchestra.run.smithi084.stdout:Installed products updated. 2023-07-06T21:38:13.910 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:13.911 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2023-07-06T21:38:13.911 INFO:teuthology.orchestra.run.smithi084.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-06T21:38:13.911 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.911 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.911 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2023-07-06T21:38:13.911 INFO:teuthology.orchestra.run.smithi084.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-06T21:38:13.912 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.1-7.el8.noarch 2023-07-06T21:38:13.912 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.0.4-10.el8.x86_64 2023-07-06T21:38:13.912 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.912 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.912 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.913 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.913 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.913 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.913 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.913 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.914 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.915 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:18.0.0-4488.g0a781ef0.el8.noarch 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.1-9.el8.x86_64 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: jq-1.6-3.el8.x86_64 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-06T21:38:13.916 INFO:teuthology.orchestra.run.smithi084.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-06T21:38:13.917 INFO:teuthology.orchestra.run.smithi084.stdout: libpq-13.5-1.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-06T21:38:13.918 INFO:teuthology.orchestra.run.smithi084.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-06T21:38:13.919 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-06T21:38:13.919 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-06T21:38:13.919 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-06T21:38:13.919 INFO:teuthology.orchestra.run.smithi084.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-06T21:38:13.919 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-06T21:38:13.920 INFO:teuthology.orchestra.run.smithi084.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-06T21:38:13.920 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-06T21:38:13.920 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-06T21:38:13.920 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-06T21:38:13.920 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-06T21:38:13.920 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-06T21:38:13.921 INFO:teuthology.orchestra.run.smithi084.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-06T21:38:13.921 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T21:38:13.921 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T21:38:13.921 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.922 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.923 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.923 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-06T21:38:13.923 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-06T21:38:13.923 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-06T21:38:13.923 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-06T21:38:13.924 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-06T21:38:13.925 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-2.6-1.el8.noarch 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-06T21:38:13.926 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-06T21:38:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-06T21:38:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-06T21:38:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-06T21:38:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.928 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.928 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-06T21:38:13.928 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-06T21:38:13.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-06T21:38:13.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-06T21:38:13.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-06T21:38:13.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-06T21:38:13.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-06T21:38:13.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-06T21:38:13.931 INFO:teuthology.orchestra.run.smithi084.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-06T21:38:13.931 INFO:teuthology.orchestra.run.smithi084.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-06T21:38:13.931 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-06T21:38:13.931 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-06T21:38:13.932 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-06T21:38:13.932 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-06T21:38:13.932 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-06T21:38:13.933 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.933 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.933 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:18.0.0-4488.g0a781ef0.el8.x86_64 2023-07-06T21:38:13.933 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-06T21:38:13.933 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-06T21:38:13.934 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-06T21:38:13.934 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-06T21:38:13.934 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-06T21:38:13.934 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-06T21:38:13.935 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:13.935 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2023-07-06T21:38:14.540 DEBUG:teuthology.parallel:result is None 2023-07-06T21:38:14.541 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0a781ef0807ae9473fb1306ef07036226ca4f1a6 2023-07-06T21:38:14.755 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T21:38:14.811 INFO:teuthology.orchestra.run.smithi084.stdout:18.0.0-4488.g0a781ef0.el8 2023-07-06T21:38:14.812 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4488.g0a781ef0.el8 2023-07-06T21:38:14.812 INFO:teuthology.task.install:The correct ceph version 18.0.0-4488.g0a781ef0 is installed. 2023-07-06T21:38:14.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0a781ef0807ae9473fb1306ef07036226ca4f1a6 2023-07-06T21:38:15.021 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-06T21:38:15.077 INFO:teuthology.orchestra.run.smithi107.stdout:18.0.0-4488.g0a781ef0.el8 2023-07-06T21:38:15.077 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4488.g0a781ef0.el8 2023-07-06T21:38:15.078 INFO:teuthology.task.install:The correct ceph version 18.0.0-4488.g0a781ef0 is installed. 2023-07-06T21:38:15.078 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-06T21:38:15.079 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:15.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T21:38:15.174 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:15.175 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-06T21:38:15.263 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-06T21:38:15.264 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:15.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T21:38:15.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T21:38:15.441 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:15.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2023-07-06T21:38:15.509 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-06T21:38:15.619 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-06T21:38:15.620 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:15.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T21:38:15.688 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T21:38:15.801 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:15.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-06T21:38:15.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-06T21:38:15.979 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-06T21:38:15.980 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:15.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T21:38:16.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T21:38:16.159 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:16.159 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2023-07-06T21:38:16.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-06T21:38:16.336 INFO:teuthology.run_tasks:Running task ceph... 2023-07-06T21:38:16.408 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-07-06T21:38:16.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /var/log/ceph 2023-07-06T21:38:16.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2023-07-06T21:38:16.480 INFO:tasks.ceph:Disabling ceph logrotate... 2023-07-06T21:38:16.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T21:38:16.484 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2023-07-06T21:38:16.553 INFO:tasks.ceph:Creating extra log directories... 2023-07-06T21:38:16.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T21:38:16.558 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-07-06T21:38:16.631 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-07-06T21:38:16.632 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-07-06T21:38:16.632 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-07-06T21:38:16.632 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:16.633 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T21:38:16.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T21:38:16.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T21:38:16.960 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:16.960 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-07-06T21:38:17.002 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-07-06T21:38:17.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-07-06T21:38:17.331 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-07-06T21:38:17.332 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-07-06T21:38:17.332 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242', 'branch': 'wip-rishabh-CephManager-in-CephFSTestCase', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm}', 'email': None, 'first_in_suite': False, 'job_id': '7328242', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-CephManager-in-CephFSTestCase'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-CephManager-in-CephFSTestCase', 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-CephManager-in-CephFSTestCase', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi084.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi107.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '84139c18d56f518615369e8310ae4ecf07a36dcd'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0a781ef0807ae9473fb1306ef07036226ca4f1a6'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'b53f236c1077f1d295baafb9e0ca9a4b447019d7', 'timestamp': '2023-07-06_19:46:43', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23459'} 2023-07-06T21:38:17.333 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T21:38:17.337 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-07-06T21:38:17.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T21:38:17.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2023-07-06T21:38:17.489 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:17.490 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T21:38:17.532 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T21:38:17.533 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2023-07-06T21:38:17.617 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T21:38:17.618 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:17.618 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-07-06T21:38:17.618 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:17.618 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:17.618 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-07-06 21:38:07.612318054 +0000 2023-07-06T21:38:17.618 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-07-06 21:35:02.119202782 +0000 2023-07-06T21:38:17.619 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-07-06 21:35:02.119202782 +0000 2023-07-06T21:38:17.619 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-07-06 21:35:02.093203219 +0000 2023-07-06T21:38:17.619 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T21:38:17.724 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-07-06T21:38:17.725 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-07-06T21:38:17.725 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000342937 s, 1.5 MB/s 2023-07-06T21:38:17.727 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T21:38:17.773 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2023-07-06T21:38:17.857 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T21:38:17.857 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:17.857 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-07-06T21:38:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-07-06 21:38:07.612318054 +0000 2023-07-06T21:38:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-07-06 21:35:02.930189146 +0000 2023-07-06T21:38:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-07-06 21:35:02.930189146 +0000 2023-07-06T21:38:17.858 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-07-06 21:35:02.905189567 +0000 2023-07-06T21:38:17.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T21:38:17.962 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-07-06T21:38:17.963 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-07-06T21:38:17.963 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000467781 s, 1.1 MB/s 2023-07-06T21:38:17.965 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T21:38:18.011 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2023-07-06T21:38:18.096 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T21:38:18.096 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:18.096 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-07-06T21:38:18.096 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:18.096 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:18.097 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-07-06 21:38:07.612318054 +0000 2023-07-06T21:38:18.097 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-07-06 21:35:03.762175158 +0000 2023-07-06T21:38:18.097 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-07-06 21:35:03.762175158 +0000 2023-07-06T21:38:18.097 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-07-06 21:35:03.737175579 +0000 2023-07-06T21:38:18.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T21:38:18.202 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-07-06T21:38:18.202 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-07-06T21:38:18.202 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000274215 s, 1.9 MB/s 2023-07-06T21:38:18.204 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T21:38:18.249 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2023-07-06T21:38:18.333 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T21:38:18.334 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:18.334 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-07-06T21:38:18.334 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:18.334 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:18.334 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2023-07-06 21:38:07.612318054 +0000 2023-07-06T21:38:18.335 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2023-07-06 21:35:04.602161036 +0000 2023-07-06T21:38:18.335 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2023-07-06 21:35:04.602161036 +0000 2023-07-06T21:38:18.335 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2023-07-06 21:35:04.575161490 +0000 2023-07-06T21:38:18.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T21:38:18.440 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2023-07-06T21:38:18.440 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2023-07-06T21:38:18.440 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000300988 s, 1.7 MB/s 2023-07-06T21:38:18.442 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T21:38:18.488 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:18.489 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:18.489 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2023-07-06T21:38:18.531 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-06T21:38:18.531 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2023-07-06T21:38:18.614 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-06T21:38:18.615 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:18.615 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 448 Links: 1 2023-07-06T21:38:18.615 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:18.615 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:18.615 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-06 21:38:05.225892285 +0000 2023-07-06T21:38:18.615 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-06 21:35:02.125606025 +0000 2023-07-06T21:38:18.616 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-06 21:35:02.125606025 +0000 2023-07-06T21:38:18.616 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-06 21:35:02.097606472 +0000 2023-07-06T21:38:18.616 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-06T21:38:18.720 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-06T21:38:18.720 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-06T21:38:18.720 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000336005 s, 1.5 MB/s 2023-07-06T21:38:18.723 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-06T21:38:18.768 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2023-07-06T21:38:18.853 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-06T21:38:18.853 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 454 Links: 1 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-06 21:38:05.226892269 +0000 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-06 21:35:02.952592824 +0000 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-06 21:35:02.952592824 +0000 2023-07-06T21:38:18.854 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-06 21:35:02.925593255 +0000 2023-07-06T21:38:18.855 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-06T21:38:18.962 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-06T21:38:18.962 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-06T21:38:18.962 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000295934 s, 1.7 MB/s 2023-07-06T21:38:18.964 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-06T21:38:19.009 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2023-07-06T21:38:19.094 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 460 Links: 1 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-06 21:38:05.226892269 +0000 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-06 21:35:03.766579831 +0000 2023-07-06T21:38:19.095 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-06 21:35:03.766579831 +0000 2023-07-06T21:38:19.096 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-06 21:35:03.739580262 +0000 2023-07-06T21:38:19.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-06T21:38:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-06T21:38:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-06T21:38:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000298781 s, 1.7 MB/s 2023-07-06T21:38:19.203 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-06T21:38:19.250 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2023-07-06T21:38:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-06T21:38:19.333 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-06T21:38:19.333 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 466 Links: 1 2023-07-06T21:38:19.334 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-06T21:38:19.334 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2023-07-06T21:38:19.334 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2023-07-06 21:38:05.226892269 +0000 2023-07-06T21:38:19.334 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2023-07-06 21:35:04.567567045 +0000 2023-07-06T21:38:19.334 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2023-07-06 21:35:04.567567045 +0000 2023-07-06T21:38:19.334 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2023-07-06 21:35:04.542567445 +0000 2023-07-06T21:38:19.334 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-06T21:38:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2023-07-06T21:38:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2023-07-06T21:38:19.438 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000313974 s, 1.6 MB/s 2023-07-06T21:38:19.440 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-06T21:38:19.486 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:19.486 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T21:38:19.487 INFO:tasks.ceph:Generating config... 2023-07-06T21:38:19.488 INFO:tasks.ceph:[client] client mount timeout = 600 2023-07-06T21:38:19.488 INFO:tasks.ceph:[client] debug client = 20 2023-07-06T21:38:19.488 INFO:tasks.ceph:[client] debug ms = 1 2023-07-06T21:38:19.489 INFO:tasks.ceph:[client] pid file = 2023-07-06T21:38:19.489 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-07-06T21:38:19.489 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-07-06T21:38:19.489 INFO:tasks.ceph:[mds] debug mds = 20 2023-07-06T21:38:19.489 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-07-06T21:38:19.489 INFO:tasks.ceph:[mds] debug ms = 1 2023-07-06T21:38:19.490 INFO:tasks.ceph:[mds] mds debug frag = True 2023-07-06T21:38:19.490 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-07-06T21:38:19.490 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-07-06T21:38:19.490 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-07-06T21:38:19.490 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-07-06T21:38:19.491 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-07-06T21:38:19.491 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-07-06T21:38:19.491 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-07-06T21:38:19.491 INFO:tasks.ceph:[mgr] debug ms = 1 2023-07-06T21:38:19.491 INFO:tasks.ceph:[mon] debug mon = 20 2023-07-06T21:38:19.492 INFO:tasks.ceph:[mon] debug ms = 1 2023-07-06T21:38:19.492 INFO:tasks.ceph:[mon] debug paxos = 20 2023-07-06T21:38:19.492 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-07-06T21:38:19.492 INFO:tasks.ceph:[osd] bdev async discard = True 2023-07-06T21:38:19.492 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-07-06T21:38:19.492 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-07-06T21:38:19.493 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-07-06T21:38:19.493 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-07-06T21:38:19.493 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-07-06T21:38:19.493 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-07-06T21:38:19.493 INFO:tasks.ceph:[osd] debug ms = 1 2023-07-06T21:38:19.494 INFO:tasks.ceph:[osd] debug osd = 20 2023-07-06T21:38:19.494 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-07-06T21:38:19.494 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-07-06T21:38:19.494 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-07-06T21:38:19.494 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-07-06T21:38:19.495 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-07-06T21:38:19.495 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-07-06T21:38:19.495 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-07-06T21:38:19.495 INFO:tasks.ceph:Setting up mon.a... 2023-07-06T21:38:19.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-07-06T21:38:19.632 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.keyring 2023-07-06T21:38:19.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-07-06T21:38:19.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T21:38:19.809 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.84'), ('mon.b', '172.21.15.107'), ('mon.c', '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]')] 2023-07-06T21:38:19.809 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.84,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-07-06T21:38:19.810 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:19.811 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T21:38:19.852 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.84 --add b 172.21.15.107 --addv c '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:19.979 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:19.979 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: generated fsid c2762452-5842-4ccc-b0b9-8dbf6ac718c9 2023-07-06T21:38:19.979 INFO:teuthology.orchestra.run.smithi084.stdout:setting min_mon_release = pacific 2023-07-06T21:38:19.979 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 0 2023-07-06T21:38:19.979 INFO:teuthology.orchestra.run.smithi084.stdout:fsid c2762452-5842-4ccc-b0b9-8dbf6ac718c9 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:last_changed 2023-07-06T21:38:19.977673+0000 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:created 2023-07-06T21:38:19.977673+0000 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:min_mon_release 16 (pacific) 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:election_strategy: 1 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2023-07-06T21:38:19.980 INFO:teuthology.orchestra.run.smithi084.stdout:2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2023-07-06T21:38:19.981 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-07-06T21:38:19.982 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-07-06T21:38:20.025 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c2762452-5842-4ccc-b0b9-8dbf6ac718c9... 2023-07-06T21:38:20.026 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T21:38:20.070 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-07-06T21:38:20.183 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2023-07-06T21:38:20.183 INFO:teuthology.orchestra.run.smithi084.stdout: chdir = "" 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: auth supported = cephx 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: filestore xattr use omap = true 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: mon clock drift allowed = 1.000 2023-07-06T21:38:20.184 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: osd crush chooseleaf type = 0 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: auth debug = true 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on old message = true 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: ms die on bug = true 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.185 INFO:teuthology.orchestra.run.smithi084.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: mon pg warn max object skew = 0 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default size = 2 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.186 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow primary affinity = true 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd allow pg remap = true 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on legacy crush tunables = false 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on crush straw calc version zero = false 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on no sortbitwise = false 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on osd down out interval zero = false 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon warn on too few osds = false 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T21:38:20.187 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T21:38:20.188 INFO:teuthology.orchestra.run.smithi084.stdout: mon_allow_pool_size_one = true 2023-07-06T21:38:20.188 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.188 INFO:teuthology.orchestra.run.smithi084.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T21:38:20.188 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: osd default data pool replay window = 5 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: mon allow pool delete = true 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: mon cluster log file level = debug 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: debug asserts on shutdown = true 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: mon health detail to clog = false 2023-07-06T21:38:20.189 INFO:teuthology.orchestra.run.smithi084.stdout: mon host = "172.21.15.84,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = c2762452-5842-4ccc-b0b9-8dbf6ac718c9 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout:[osd] 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: osd journal size = 100 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub load threshold = 5.0 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: osd scrub max interval = 600 2023-07-06T21:38:20.190 INFO:teuthology.orchestra.run.smithi084.stdout: osd mclock profile = high_recovery_ops 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: osd recover clone overlap = true 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: osd recovery max chunk = 1048576 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug shutdown = true 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug op order = true 2023-07-06T21:38:20.191 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug verify stray on activate = true 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: osd open classes on start = true 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug pg log writeout = true 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: osd deep scrub update digest min age = 30 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: osd map max advance = 10 2023-07-06T21:38:20.192 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: journal zero on create = true 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: filestore ondisk finisher threads = 3 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: filestore apply finisher threads = 3 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: bdev debug aio = true 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: osd debug misdirected ops = true 2023-07-06T21:38:20.193 INFO:teuthology.orchestra.run.smithi084.stdout: bdev async discard = True 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: bdev enable discard = True 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore allocator = bitmap 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore block size = 96636764160 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: bluestore fsck on mount = True 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluefs = 1/20 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: debug bluestore = 1/20 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-07-06T21:38:20.194 INFO:teuthology.orchestra.run.smithi084.stdout: debug osd = 20 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: debug rocksdb = 4/10 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd full ratio = 0.9 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd nearfull ratio = 0.8 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: osd failsafe full ratio = 0.95 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: osd objectstore = bluestore 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2023-07-06T21:38:20.195 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr] 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: debug mgr = 20 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2023-07-06T21:38:20.196 INFO:teuthology.orchestra.run.smithi084.stdout: mgr/telemetry/nag = false 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout:[mon] 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: debug mon = 20 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: debug paxos = 20 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: debug auth = 20 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: mon data avail warn = 5 2023-07-06T21:38:20.197 INFO:teuthology.orchestra.run.smithi084.stdout: mon mgr mkfs grace = 240 2023-07-06T21:38:20.198 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min pgs per osd = 4 2023-07-06T21:38:20.198 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd reporter subtree level = osd 2023-07-06T21:38:20.198 INFO:teuthology.orchestra.run.smithi084.stdout: mon osd prime pg temp = true 2023-07-06T21:38:20.198 INFO:teuthology.orchestra.run.smithi084.stdout: mon reweight min bytes per osd = 10 2023-07-06T21:38:20.198 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T21:38:20.199 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: # 1m isn't quite enough 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: mon_down_mkfs_grace = 2m 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: mon_warn_on_filestore_osds = false 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: mon op complaint time = 120 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:38:20.200 INFO:teuthology.orchestra.run.smithi084.stdout:[client] 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: rgw cache enabled = true 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable ops log = true 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: rgw enable usage log = true 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: client mount timeout = 600 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: debug client = 20 2023-07-06T21:38:20.201 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout: pid file = "" 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.a] 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.b] 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout:[mon.c] 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2023-07-06T21:38:20.202 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout:[mds] 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds = 20 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout: debug mds balancer = 20 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout: debug ms = 1 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug frag = True 2023-07-06T21:38:20.203 INFO:teuthology.orchestra.run.smithi084.stdout: mds debug scatterstat = True 2023-07-06T21:38:20.204 INFO:teuthology.orchestra.run.smithi084.stdout: mds op complaint time = 180 2023-07-06T21:38:20.204 INFO:teuthology.orchestra.run.smithi084.stdout: mds verify scatter = True 2023-07-06T21:38:20.204 INFO:teuthology.orchestra.run.smithi084.stdout: osd op complaint time = 180 2023-07-06T21:38:20.204 INFO:teuthology.orchestra.run.smithi084.stdout: rados mon op timeout = 900 2023-07-06T21:38:20.204 INFO:teuthology.orchestra.run.smithi084.stdout: rados osd op timeout = 900 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2023-07-06T21:38:20.205 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2023-07-06T21:38:20.206 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.207 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2023-07-06T21:38:20.208 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.208 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2023-07-06T21:38:20.208 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2023-07-06T21:38:20.208 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2023-07-06T21:38:20.208 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2023-07-06T21:38:20.208 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2023-07-06T21:38:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2023-07-06T21:38:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2023-07-06T21:38:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-07-06T21:38:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2023-07-06T21:38:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2023-07-06T21:38:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2023-07-06T21:38:20.210 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.84,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = c2762452-5842-4ccc-b0b9-8dbf6ac718c9 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.211 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2023-07-06T21:38:20.212 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.213 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2023-07-06T21:38:20.214 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: bdev async discard = True 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: bdev enable discard = True 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore allocator = bitmap 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 1/20 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 1/20 2023-07-06T21:38:20.215 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 4/10 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2023-07-06T21:38:20.216 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2023-07-06T21:38:20.217 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2023-07-06T21:38:20.218 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2023-07-06T21:38:20.219 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2023-07-06T21:38:20.219 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2023-07-06T21:38:20.219 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2023-07-06T21:38:20.219 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2023-07-06T21:38:20.219 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-07-06T21:38:20.220 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout: 2023-07-06T21:38:20.221 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2023-07-06T21:38:20.222 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = "" 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2023-07-06T21:38:20.223 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds balancer = 20 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2023-07-06T21:38:20.224 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2023-07-06T21:38:20.225 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2023-07-06T21:38:20.225 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2023-07-06T21:38:20.225 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2023-07-06T21:38:20.225 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2023-07-06T21:38:20.225 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2023-07-06T21:38:20.226 INFO:tasks.ceph:Creating admin key on mon.a... 2023-07-06T21:38:20.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-07-06T21:38:20.341 INFO:tasks.ceph:Copying monmap to all nodes... 2023-07-06T21:38:20.341 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:20.342 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-07-06T21:38:20.383 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:20.383 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-07-06T21:38:20.465 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:38:20.466 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:20.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T21:38:20.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T21:38:20.601 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:20.601 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:20.685 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:38:20.685 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:20.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2023-07-06T21:38:20.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-07-06T21:38:20.783 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:20.783 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:20.866 INFO:tasks.ceph:Setting up mon nodes... 2023-07-06T21:38:20.867 INFO:tasks.ceph:Setting up mgr nodes... 2023-07-06T21:38:20.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T21:38:20.994 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-07-06T21:38:21.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T21:38:21.162 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-07-06T21:38:21.169 INFO:tasks.ceph:Setting up mds nodes... 2023-07-06T21:38:21.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T21:38:21.298 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-07-06T21:38:21.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-07-06T21:38:21.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T21:38:21.496 INFO:teuthology.orchestra.run.smithi084.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-07-06T21:38:21.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-07-06T21:38:21.570 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T21:38:21.696 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-07-06T21:38:21.702 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-07-06T21:38:21.771 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T21:38:21.898 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-07-06T21:38:21.903 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-07-06T21:38:21.971 INFO:tasks.ceph_client:Setting up client nodes... 2023-07-06T21:38:21.972 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-06T21:38:22.070 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-07-06T21:38:22.104 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-07-06T21:38:22.199 INFO:teuthology.orchestra.run.smithi084.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-07-06T21:38:22.232 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-07-06T21:38:22.331 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-07-06T21:38:22.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-07-06T21:38:22.460 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-07-06T21:38:22.493 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-07-06T21:38:22.493 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi084.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-07-06T21:38:22.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-07-06T21:38:22.561 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:22.561 INFO:tasks.ceph:role: osd.0 2023-07-06T21:38:22.561 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:38:22.562 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T21:38:22.637 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:22.637 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:22.637 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:22.637 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:22.638 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:22.638 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:22.638 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:22.638 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:22.638 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:22.638 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:23.068 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-07-06T21:38:23.072 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-07-06T21:38:23.073 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-07-06T21:38:23.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-07-06T21:38:23.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-07-06T21:38:23.396 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:23.397 INFO:tasks.ceph:role: osd.1 2023-07-06T21:38:23.397 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:38:23.397 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T21:38:23.473 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:23.474 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:23.474 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:23.474 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:23.474 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:23.475 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:23.475 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:23.475 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:23.475 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:23.475 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:23.880 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-07-06T21:38:23.885 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-07-06T21:38:23.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-07-06T21:38:24.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-07-06T21:38:24.092 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-07-06T21:38:24.162 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:24.163 INFO:tasks.ceph:role: osd.2 2023-07-06T21:38:24.163 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:38:24.163 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T21:38:24.237 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:24.238 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:24.238 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:24.238 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:24.238 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:24.239 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:24.239 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:24.239 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:24.239 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:24.239 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:24.680 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-07-06T21:38:24.684 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-07-06T21:38:24.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-07-06T21:38:24.843 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-07-06T21:38:24.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-07-06T21:38:24.983 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:24.983 INFO:tasks.ceph:role: osd.3 2023-07-06T21:38:24.984 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:38:24.984 DEBUG:teuthology.orchestra.run.smithi084:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T21:38:25.058 INFO:teuthology.orchestra.run.smithi084.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:25.058 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:25.058 INFO:teuthology.orchestra.run.smithi084.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:25.059 INFO:teuthology.orchestra.run.smithi084.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:25.494 INFO:teuthology.orchestra.run.smithi084.stdout:Discarding blocks...Done. 2023-07-06T21:38:25.499 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi084.front.sepia.ceph.com -o noatime 2023-07-06T21:38:25.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-07-06T21:38:25.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-07-06T21:38:25.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:25.855+0000 7f1c35a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-07-06T21:38:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:25.856+0000 7f1c35a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-07-06T21:38:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:25.856+0000 7f1c35a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T21:38:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:25.856+0000 7f1c35a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T21:38:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:25.856+0000 7f1c35a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-07-06T21:38:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:25.858+0000 7f1c35a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-07-06T21:38:28.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-07-06T21:38:28.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:28.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:28.980+0000 7f182857a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-07-06T21:38:28.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:28.980+0000 7f182857a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-07-06T21:38:28.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:28.980+0000 7f182857a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T21:38:28.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:28.980+0000 7f182857a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T21:38:28.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:28.980+0000 7f182857a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-07-06T21:38:28.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:28.982+0000 7f182857a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-07-06T21:38:31.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-07-06T21:38:31.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:32.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:32.080+0000 7fdc442d1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-07-06T21:38:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:32.080+0000 7fdc442d1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-07-06T21:38:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:32.081+0000 7fdc442d1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T21:38:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:32.081+0000 7fdc442d1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T21:38:32.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:32.081+0000 7fdc442d1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-07-06T21:38:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:32.082+0000 7fdc442d1640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-07-06T21:38:35.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-07-06T21:38:35.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:35.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:35.192+0000 7fa42d368640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-07-06T21:38:35.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:35.192+0000 7fa42d368640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-07-06T21:38:35.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:35.192+0000 7fa42d368640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T21:38:35.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:35.193+0000 7fa42d368640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T21:38:35.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:35.193+0000 7fa42d368640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-07-06T21:38:35.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:35.194+0000 7fa42d368640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-07-06T21:38:38.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-07-06T21:38:38.191 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-07-06T21:38:38.260 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:38.260 INFO:tasks.ceph:role: osd.4 2023-07-06T21:38:38.260 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:38:38.261 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-07-06T21:38:38.335 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:38.335 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:38.335 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:38.335 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:38.336 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:38.336 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:38.336 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:38.336 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:38.336 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:38.336 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:38.768 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-06T21:38:38.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-06T21:38:38.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-07-06T21:38:38.975 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-07-06T21:38:39.055 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-07-06T21:38:39.125 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:39.125 INFO:tasks.ceph:role: osd.5 2023-07-06T21:38:39.126 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:38:39.126 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-07-06T21:38:39.200 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:39.200 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:39.201 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:39.607 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-06T21:38:39.624 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-06T21:38:39.625 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-07-06T21:38:39.757 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-07-06T21:38:39.834 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-07-06T21:38:39.901 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:39.902 INFO:tasks.ceph:role: osd.6 2023-07-06T21:38:39.902 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:38:39.902 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-07-06T21:38:39.979 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:39.979 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:39.980 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:39.980 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:39.980 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:39.980 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:39.981 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:39.981 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:39.981 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:39.981 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:40.416 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-06T21:38:40.421 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-06T21:38:40.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-07-06T21:38:40.595 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-07-06T21:38:40.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-07-06T21:38:40.741 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-07-06T21:38:40.741 INFO:tasks.ceph:role: osd.7 2023-07-06T21:38:40.741 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:38:40.742 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-07-06T21:38:40.818 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-07-06T21:38:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2023-07-06T21:38:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-07-06T21:38:40.819 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-07-06T21:38:40.819 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-07-06T21:38:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2023-07-06T21:38:40.820 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-07-06T21:38:40.820 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-07-06T21:38:40.820 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-07-06T21:38:40.821 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-07-06T21:38:41.253 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2023-07-06T21:38:41.258 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2023-07-06T21:38:41.258 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-07-06T21:38:41.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-07-06T21:38:41.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:41.596 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:41.594+0000 7f9064e2b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-07-06T21:38:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:41.595+0000 7f9064e2b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-07-06T21:38:41.597 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:41.595+0000 7f9064e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T21:38:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:41.595+0000 7f9064e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T21:38:41.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:41.595+0000 7f9064e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-07-06T21:38:41.599 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:41.597+0000 7f9064e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-07-06T21:38:44.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-07-06T21:38:44.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:44.692 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:44.689+0000 7f7d55e7f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-07-06T21:38:44.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:44.690+0000 7f7d55e7f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-07-06T21:38:44.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:44.690+0000 7f7d55e7f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T21:38:44.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:44.690+0000 7f7d55e7f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T21:38:44.693 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:44.690+0000 7f7d55e7f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-07-06T21:38:44.694 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:44.692+0000 7f7d55e7f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-07-06T21:38:47.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:38:47.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:38:47.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-07-06T21:38:47.714 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:47.820 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:47.818+0000 7f4ebd5e2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-07-06T21:38:47.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:47.819+0000 7f4ebd5e2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-07-06T21:38:47.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:47.820+0000 7f4ebd5e2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T21:38:47.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:47.820+0000 7f4ebd5e2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T21:38:47.821 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:47.820+0000 7f4ebd5e2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-07-06T21:38:47.822 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:47.821+0000 7f4ebd5e2640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-07-06T21:38:50.777 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-07-06T21:38:50.845 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:50.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:50.949+0000 7fd3fd992640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-07-06T21:38:50.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:50.949+0000 7fd3fd992640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-07-06T21:38:50.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:50.949+0000 7fd3fd992640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T21:38:50.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:50.949+0000 7fd3fd992640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T21:38:50.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:50.949+0000 7fd3fd992640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-07-06T21:38:50.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:50.951+0000 7fd3fd992640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-07-06T21:38:53.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-07-06T21:38:53.949 INFO:tasks.ceph:Reading keys from all nodes... 2023-07-06T21:38:53.950 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:53.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-07-06T21:38:54.019 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:54.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-07-06T21:38:54.128 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:54.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-07-06T21:38:54.242 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:54.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-07-06T21:38:54.353 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:54.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-07-06T21:38:54.460 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:54.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-07-06T21:38:54.567 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:54.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-07-06T21:38:54.673 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:54.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-07-06T21:38:54.744 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:54.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-07-06T21:38:54.851 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:54.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-07-06T21:38:54.958 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:54.959 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-07-06T21:38:55.067 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:55.068 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-07-06T21:38:55.177 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:55.177 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-07-06T21:38:55.287 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:55.287 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-07-06T21:38:55.395 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:55.395 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-07-06T21:38:55.437 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:55.437 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-07-06T21:38:55.521 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:55.522 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-07-06T21:38:55.563 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:55.564 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-07-06T21:38:55.646 INFO:tasks.ceph:Adding keys to all mons... 2023-07-06T21:38:55.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T21:38:55.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo tee -a /etc/ceph/ceph.keyring 2023-07-06T21:38:55.713 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.y] 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBMNKdkfcUzOxAA9BUPVthEZwBTmBTqdmDopg== 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a] 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBNNKdk+/m1ERAAVuVl/rTq2tc0Dhv83fjupA== 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c] 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBNNKdkpgiGHRAA+dmQ4tCjJggxC0UEAx4E8A== 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.0] 2023-07-06T21:38:55.714 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBRNKdko5ENMxAAkg9Bb14zud5gryZG5cNFmA== 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.1] 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBUNKdkIAyGOhAAGm6K8mPZUi+KIOhlHHcf/A== 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.2] 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBYNKdkPYfnBBAAQiGXT+pTVZ/rM8G100/dGQ== 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.3] 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBbNKdklfeTCxAAEMCI574RRSmAnzz2hU00Jg== 2023-07-06T21:38:55.715 INFO:teuthology.orchestra.run.smithi084.stdout:[mgr.x] 2023-07-06T21:38:55.716 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBNNKdk8JijCRAAGnCGHPatWKPv0GPKCjXNtg== 2023-07-06T21:38:55.716 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b] 2023-07-06T21:38:55.716 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBNNKdkRtxvKRAAIt8IaHnqNpNdKNcNdnoN7g== 2023-07-06T21:38:55.716 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d] 2023-07-06T21:38:55.716 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBNNKdkdtB1NRAAZPgJEifPgIZTnZI5Sr0ZIw== 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.4] 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBhNKdkluR8IxAAd2WQtPkTN09bIdqHrbRBCg== 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.5] 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBkNKdkH7wwKRAAyXE0BPwP8/koUUWCcCA1dQ== 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.6] 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBnNKdk363RMBAArv8mMHQtqZRUf3AN2PqwTQ== 2023-07-06T21:38:55.717 INFO:teuthology.orchestra.run.smithi084.stdout:[osd.7] 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBqNKdkTpaXOBAACmQ850h7EOrJ8sl0ODNYmA== 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBONKdkqmQnBBAALedUX3/ZDQ9mAhOGEH6QrA== 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout:[client.2] 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBONKdk0bfQCxAA+4nQgIo/iEDM930/C0JV1w== 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBONKdkLt+uExAAzCoF/xhx1jkry5bkwObTEQ== 2023-07-06T21:38:55.718 INFO:teuthology.orchestra.run.smithi084.stdout:[client.3] 2023-07-06T21:38:55.719 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQBONKdkodhbGxAA8n9L0VrpLjBlrDyEi4Sztg== 2023-07-06T21:38:55.752 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.y] 2023-07-06T21:38:55.753 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBMNKdkfcUzOxAA9BUPVthEZwBTmBTqdmDopg== 2023-07-06T21:38:55.753 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2023-07-06T21:38:55.753 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBNNKdk+/m1ERAAVuVl/rTq2tc0Dhv83fjupA== 2023-07-06T21:38:55.753 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2023-07-06T21:38:55.753 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBNNKdkpgiGHRAA+dmQ4tCjJggxC0UEAx4E8A== 2023-07-06T21:38:55.753 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.0] 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBRNKdko5ENMxAAkg9Bb14zud5gryZG5cNFmA== 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.1] 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBUNKdkIAyGOhAAGm6K8mPZUi+KIOhlHHcf/A== 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.2] 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBYNKdkPYfnBBAAQiGXT+pTVZ/rM8G100/dGQ== 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.3] 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBbNKdklfeTCxAAEMCI574RRSmAnzz2hU00Jg== 2023-07-06T21:38:55.754 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.x] 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBNNKdk8JijCRAAGnCGHPatWKPv0GPKCjXNtg== 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBNNKdkRtxvKRAAIt8IaHnqNpNdKNcNdnoN7g== 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBNNKdkdtB1NRAAZPgJEifPgIZTnZI5Sr0ZIw== 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.4] 2023-07-06T21:38:55.755 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBhNKdkluR8IxAAd2WQtPkTN09bIdqHrbRBCg== 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.5] 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBkNKdkH7wwKRAAyXE0BPwP8/koUUWCcCA1dQ== 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.6] 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBnNKdk363RMBAArv8mMHQtqZRUf3AN2PqwTQ== 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.7] 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBqNKdkTpaXOBAACmQ850h7EOrJ8sl0ODNYmA== 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2023-07-06T21:38:55.756 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBONKdkqmQnBBAALedUX3/ZDQ9mAhOGEH6QrA== 2023-07-06T21:38:55.757 INFO:teuthology.orchestra.run.smithi107.stdout:[client.2] 2023-07-06T21:38:55.757 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBONKdk0bfQCxAA+4nQgIo/iEDM930/C0JV1w== 2023-07-06T21:38:55.757 INFO:teuthology.orchestra.run.smithi107.stdout:[client.1] 2023-07-06T21:38:55.757 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBONKdkLt+uExAAzCoF/xhx1jkry5bkwObTEQ== 2023-07-06T21:38:55.757 INFO:teuthology.orchestra.run.smithi107.stdout:[client.3] 2023-07-06T21:38:55.757 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBONKdkodhbGxAA8n9L0VrpLjBlrDyEi4Sztg== 2023-07-06T21:38:55.758 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T21:38:55.764 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T21:38:55.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:55.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.021 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.251 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.364 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.479 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T21:38:56.590 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-07-06T21:38:56.697 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.701 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.818 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-07-06T21:38:56.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:56.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.046 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.159 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-07-06T21:38:57.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.401 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.516 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.741 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-07-06T21:38:57.849 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-07-06T21:38:57.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-07-06T21:38:57.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T21:38:58.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-07-06T21:38:58.110 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-07-06T21:38:58.179 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T21:38:58.299 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-07-06T21:38:58.367 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-07-06T21:38:58.434 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-07-06T21:38:58.553 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-07-06T21:38:58.620 DEBUG:teuthology.orchestra.run.smithi084:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:58.624 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-07-06T21:38:58.665 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-07-06T21:38:58.666 INFO:tasks.ceph.mon.a:Restarting daemon 2023-07-06T21:38:58.666 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-07-06T21:38:58.706 INFO:tasks.ceph.mon.a:Started 2023-07-06T21:38:58.707 INFO:tasks.ceph.mon.b:Restarting daemon 2023-07-06T21:38:58.707 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-07-06T21:38:58.711 INFO:tasks.ceph.mon.b:Started 2023-07-06T21:38:58.712 INFO:tasks.ceph.mon.c:Restarting daemon 2023-07-06T21:38:58.712 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-07-06T21:38:58.716 INFO:tasks.ceph.mon.c:Started 2023-07-06T21:38:58.716 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-07-06T21:38:58.716 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-07-06T21:38:58.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-07-06T21:38:58.721 INFO:tasks.ceph.mgr.y:Started 2023-07-06T21:38:58.721 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-07-06T21:38:58.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-07-06T21:38:58.726 INFO:tasks.ceph.mgr.x:Started 2023-07-06T21:38:58.726 INFO:tasks.ceph:Setting crush tunables to default 2023-07-06T21:38:58.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd crush tunables default 2023-07-06T21:38:58.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.933+0000 7f6d215ff700 1 Processor -- start 2023-07-06T21:38:58.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.933+0000 7f6d215ff700 1 -- start start 2023-07-06T21:38:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d215ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d1c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d215ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0d2b10 con 0x7f6d1c0cee50 2023-07-06T21:38:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d215ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0d0e70 con 0x7f6d1c15cc50 2023-07-06T21:38:58.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d215ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0d27e0 con 0x7f6d1c15d730 2023-07-06T21:38:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d1c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d1c0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:33976/0 (socket says 172.21.15.84:33976) 2023-07-06T21:38:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 learned_addr learned my addr 172.21.15.84:0/1348649103 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:38:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 msgr2=0x7f6d1c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-07-06T21:38:58.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 msgr2=0x7f6d1c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-07-06T21:38:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d1c0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-06T21:38:58.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.935+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d1c0cf570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-06T21:38:58.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972174434 0 0) 0x7f6d1c0d2b10 con 0x7f6d1c0cee50 2023-07-06T21:38:58.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d080035b0 con 0x7f6d1c0cee50 2023-07-06T21:38:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636719684 0 0) 0x7f6d080035b0 con 0x7f6d1c0cee50 2023-07-06T21:38:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 msgr2=0x7f6d1c0cf570 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:38:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d1c0cf570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 >> v1:172.21.15.107:6789/0 conn(0x7f6d1c15cc50 legacy=0x7f6d1c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0cd310 con 0x7f6d1c0cee50 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f6d04001670 con 0x7f6d1c0cee50 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 >> v1:172.21.15.84:6789/0 conn(0x7f6d1c0cee50 legacy=0x7f6d1c15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d08003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d080040b0 0x7f6d08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d08004eb0 0x7f6d08005320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0d27e0 con 0x7f6d08004eb0 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0800bd10 con 0x7f6d080040b0 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.941+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0800bed0 con 0x7f6d1c15d730 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d08004eb0 0x7f6d08005320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d13fff700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d080040b0 0x7f6d08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1affd700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d08003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 msgr2=0x7f6d08003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:38:58.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d1c15d730 0x7f6d08003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:58.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d080040b0 msgr2=0x7f6d08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:38:58.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d080040b0 0x7f6d08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:58.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080035b0 con 0x7f6d08004eb0 2023-07-06T21:38:58.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.942+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d08004eb0 0x7f6d08005320 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009cc0 tx=0x7f6d08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50a3a1790b29ec7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:38:58.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.943+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14017070 con 0x7f6d08004eb0 2023-07-06T21:38:58.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.943+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1400f040 con 0x7f6d08004eb0 2023-07-06T21:38:58.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.943+0000 7f6d1a7fc700 1 -- 172.21.15.84:0/1348649103 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14003c60 con 0x7f6d08004eb0 2023-07-06T21:38:58.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.943+0000 7f6d215ff700 1 -- 172.21.15.84:0/1348649103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d08004eb0 msgr2=0x7f6d08005320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:58.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.943+0000 7f6d215ff700 1 --2- 172.21.15.84:0/1348649103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d08004eb0 0x7f6d08005320 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6d14009cc0 tx=0x7f6d08005880 comp rx=0 tx=0).stop 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.943+0000 7f6d1b7fe700 1 -- 172.21.15.84:0/1348649103 reap_dead start 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.944+0000 7f6d215ff700 1 -- 172.21.15.84:0/1348649103 shutdown_connections 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.944+0000 7f6d215ff700 1 -- 172.21.15.84:0/1348649103 >> 172.21.15.84:0/1348649103 conn(0x7f6d1c0715c0 msgr2=0x7f6d1c0719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.944+0000 7f6d215ff700 1 -- 172.21.15.84:0/1348649103 shutdown_connections 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.944+0000 7f6d215ff700 1 -- 172.21.15.84:0/1348649103 wait complete. 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.946+0000 7f6d215ff700 1 Processor -- start 2023-07-06T21:38:58.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.946+0000 7f6d215ff700 1 -- start start 2023-07-06T21:38:58.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d215ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d08004eb0 0x7f6d1c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d215ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 0x7f6d1c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d215ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0c5f50 0x7f6d1c1f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d215ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c149200 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d215ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b87f0 con 0x7f6d1c0c5f50 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d215ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c6420 con 0x7f6d08004eb0 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 0x7f6d1c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 0x7f6d1c0c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44524/0 (socket says 172.21.15.84:44524) 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 -- 172.21.15.84:0/477452069 learned_addr learned my addr 172.21.15.84:0/477452069 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:38:58.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d1affd700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d08004eb0 0x7f6d1c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d1b7fe700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0c5f50 0x7f6d1c1f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 -- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d08004eb0 msgr2=0x7f6d1c0c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d08004eb0 0x7f6d1c0c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 -- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0c5f50 msgr2=0x7f6d1c1f59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0c5f50 0x7f6d1c1f59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d13fff700 1 -- 172.21.15.84:0/477452069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0b8230 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.947+0000 7f6d1affd700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d08004eb0 0x7f6d1c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:38:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d13fff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 0x7f6d1c0c5a10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6d040017a0 tx=0x7f6d0400fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400fe80 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d040052e0 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c0b8510 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d04003680 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c0c9440 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6d04003ae0 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.948+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6d0401d460 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.949+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c15e300 con 0x7f6d1c0c55a0 2023-07-06T21:38:58.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:58.952+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6d0400ddc0 con 0x7f6d1c0c55a0 2023-07-06T21:38:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.031+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6d1c0672c0 con 0x7f6d1c0c55a0 2023-07-06T21:38:59.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.034+0000 7f6d137fe700 1 -- 172.21.15.84:0/477452069 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6d0400d640 con 0x7f6d1c0c55a0 2023-07-06T21:38:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 msgr2=0x7f6d1c0c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 0x7f6d1c0c5a10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6d040017a0 tx=0x7f6d0400fc40 comp rx=0 tx=0).stop 2023-07-06T21:38:59.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 shutdown_connections 2023-07-06T21:38:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d1c0c5f50 0x7f6d1c1f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d08004eb0 0x7f6d1c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 --2- 172.21.15.84:0/477452069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d1c0c55a0 0x7f6d1c0c5a10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 >> 172.21.15.84:0/477452069 conn(0x7f6d1c0715c0 msgr2=0x7f6d1c0b1930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:38:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 shutdown_connections 2023-07-06T21:38:59.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:38:59.035+0000 7f6d215ff700 1 -- 172.21.15.84:0/477452069 wait complete. 2023-07-06T21:38:59.039 INFO:teuthology.orchestra.run.smithi084.stderr:adjusted tunables profile to default 2023-07-06T21:38:59.055 INFO:tasks.ceph:check_enable_crimson: False 2023-07-06T21:38:59.055 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-07-06T21:38:59.055 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:59.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-07-06T21:38:59.125 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:59.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-07-06T21:38:59.220 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:38:59.219+0000 7f1cd3851200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T21:38:59.233 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:59.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-07-06T21:38:59.247 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:38:59.245+0000 7f2aaff32200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-06T21:38:59.301 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:38:59.300+0000 7f1cd3851200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T21:38:59.319 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:38:59.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-07-06T21:38:59.341 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:38:59.339+0000 7f2aaff32200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-06T21:38:59.396 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:38:59.395+0000 7f1cd3851200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T21:38:59.416 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:59.417 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-07-06T21:38:59.447 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:38:59.445+0000 7f2aaff32200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-06T21:38:59.485 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:59.485 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-07-06T21:38:59.499 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:38:59.498+0000 7f1cd3851200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T21:38:59.549 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:38:59.548+0000 7f2aaff32200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-06T21:38:59.566 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:59.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-07-06T21:38:59.680 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:38:59.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-07-06T21:38:59.690 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:38:59.689+0000 7f1cd3851200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T21:38:59.737 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:38:59.735+0000 7f2aaff32200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-06T21:38:59.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 972346d6-6871-4d63-9180-095dd8ba1fed 0 2023-07-06T21:38:59.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.986+0000 7f4508fff700 1 Processor -- start 2023-07-06T21:38:59.989 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.987+0000 7f4508fff700 1 -- start start 2023-07-06T21:38:59.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4508fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 0x7f45040cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:59.990 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4508fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 0x7f45040cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4508fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040d1830 con 0x7f45040b1a80 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4508fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040cf330 con 0x7f45040ceda0 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4508fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d1270 con 0x7f45040b81c0 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f450259c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 0x7f45040cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4502d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 0x7f45040cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4502d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 0x7f45040cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46726/0 (socket says 172.21.15.107:46726) 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f450259c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 0x7f45040cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41756/0 (socket says 172.21.15.107:41756) 2023-07-06T21:38:59.991 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f4502d9d700 1 -- 172.21.15.107:0/104632818 learned_addr learned my addr 172.21.15.107:0/104632818 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:38:59.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f450259c700 1 -- 172.21.15.107:0/104632818 >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 msgr2=0x7f45040cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:38:59.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f450259c700 1 --2- 172.21.15.107:0/104632818 >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 0x7f45040cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.988+0000 7f450259c700 1 -- 172.21.15.107:0/104632818 >> v1:172.21.15.107:6789/0 conn(0x7f45040b1a80 legacy=0x7f450415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:59.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.989+0000 7f450259c700 1 -- 172.21.15.107:0/104632818 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040d1550 con 0x7f45040b81c0 2023-07-06T21:38:59.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.989+0000 7f4502d9d700 1 --2- 172.21.15.107:0/104632818 >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 0x7f45040cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:38:59.992 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.989+0000 7f450259c700 1 --2- 172.21.15.107:0/104632818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 0x7f45040cf600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009840 tx=0x7f44f4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c166e1b5907f12 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:38:59.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.989+0000 7f4501d9b700 1 -- 172.21.15.107:0/104632818 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4011830 con 0x7f45040b81c0 2023-07-06T21:38:59.993 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.989+0000 7f4501d9b700 1 -- 172.21.15.107:0/104632818 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4011990 con 0x7f45040b81c0 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.989+0000 7f4501d9b700 1 -- 172.21.15.107:0/104632818 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4011d20 con 0x7f45040b81c0 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 -- 172.21.15.107:0/104632818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 msgr2=0x7f45040cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 --2- 172.21.15.107:0/104632818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 0x7f45040cf600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009840 tx=0x7f44f4009b50 comp rx=0 tx=0).stop 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 -- 172.21.15.107:0/104632818 shutdown_connections 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 --2- 172.21.15.107:0/104632818 >> v2:172.21.15.107:3300/0 conn(0x7f45040ceda0 0x7f45040cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 --2- 172.21.15.107:0/104632818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040b81c0 0x7f45040cf600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.994 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 -- 172.21.15.107:0/104632818 >> 172.21.15.107:0/104632818 conn(0x7f450406b2b0 msgr2=0x7f450406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:38:59.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 -- 172.21.15.107:0/104632818 shutdown_connections 2023-07-06T21:38:59.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.990+0000 7f4508fff700 1 -- 172.21.15.107:0/104632818 wait complete. 2023-07-06T21:38:59.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.991+0000 7f4508fff700 1 Processor -- start 2023-07-06T21:38:59.995 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.992+0000 7f4508fff700 1 -- start start 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.992+0000 7f4508fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 0x7f45040c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f4508fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45040b81c0 0x7f45041e6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f4508fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040ceda0 0x7f45041e4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f4508fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45041e6e70 con 0x7f45040b81c0 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f4508fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45041e7cb0 con 0x7f45040b1a80 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f4508fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45041e70a0 con 0x7f45040ceda0 2023-07-06T21:38:59.997 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 0x7f45040c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 0x7f45040c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46728/0 (socket says 172.21.15.107:46728) 2023-07-06T21:38:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 -- 172.21.15.107:0/3009610192 learned_addr learned my addr 172.21.15.107:0/3009610192 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:38:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f4502d9d700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040ceda0 0x7f45041e4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:38:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 -- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040ceda0 msgr2=0x7f45041e4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040ceda0 0x7f45041e4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.998 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 -- 172.21.15.107:0/3009610192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45040b81c0 msgr2=0x7f45041e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45040b81c0 0x7f45041e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 -- 172.21.15.107:0/3009610192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009b30 con 0x7f45040b1a80 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.993+0000 7f450259c700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 0x7f45040c7ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009fd0 tx=0x7f44f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f40036a0 con 0x7f45040b1a80 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f401f070 con 0x7f45040b1a80 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4003a60 con 0x7f45040b1a80 2023-07-06T21:38:59.999 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f40094a0 con 0x7f45040b1a80 2023-07-06T21:39:00.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45041e7660 con 0x7f45040b1a80 2023-07-06T21:39:00.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f44f4026480 con 0x7f45040b1a80 2023-07-06T21:39:00.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.994+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f44f4021070 con 0x7f45040b1a80 2023-07-06T21:39:00.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.995+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450404e1d0 con 0x7f45040b1a80 2023-07-06T21:39:00.000 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:38:59.998+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f44f402b020 con 0x7f45040b1a80 2023-07-06T21:39:00.087 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.086+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "972346d6-6871-4d63-9180-095dd8ba1fed", "id": 0} v 0) v1 -- 0x7f45040b4f30 con 0x7f45040b1a80 2023-07-06T21:39:00.101 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.093+0000 7f44faffd700 1 -- 172.21.15.107:0/3009610192 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "972346d6-6871-4d63-9180-095dd8ba1fed", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44f4015020 con 0x7f45040b1a80 2023-07-06T21:39:00.101 INFO:teuthology.orchestra.run.smithi107.stdout:0 2023-07-06T21:39:00.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 msgr2=0x7f45040c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 0x7f45040c7ac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009fd0 tx=0x7f44f4004750 comp rx=0 tx=0).stop 2023-07-06T21:39:00.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 shutdown_connections 2023-07-06T21:39:00.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45040ceda0 0x7f45041e4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.102 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45040b1a80 0x7f45040c7ac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 --2- 172.21.15.107:0/3009610192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45040b81c0 0x7f45041e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 >> 172.21.15.107:0/3009610192 conn(0x7f450406b2b0 msgr2=0x7f45040b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 shutdown_connections 2023-07-06T21:39:00.103 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.094+0000 7f4508fff700 1 -- 172.21.15.107:0/3009610192 wait complete. 2023-07-06T21:39:00.112 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:00.110+0000 7f1cd3851200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T21:39:00.113 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 42a1f42a-4cec-4d2f-8943-efd17495b1c1 1 2023-07-06T21:39:00.166 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:00.164+0000 7f2aaff32200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-06T21:39:00.211 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:00.209+0000 7f1cd3851200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T21:39:00.266 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:00.264+0000 7f2aaff32200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-06T21:39:00.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.302+0000 7f18bf5ff700 1 Processor -- start 2023-07-06T21:39:00.304 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.303+0000 7f18bf5ff700 1 -- start start 2023-07-06T21:39:00.305 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bf5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f18b80b1a80 0x7f18b80d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bf5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b80d1830 con 0x7f18b80b81c0 2023-07-06T21:39:00.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bf5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b80cf330 con 0x7f18b80cfd10 2023-07-06T21:39:00.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bf5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b80d1270 con 0x7f18b80b1a80 2023-07-06T21:39:00.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bdb9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f18b80b1a80 0x7f18b80d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bcb9a700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f18b80cfd10 0x7f18b80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:34046/0 (socket says 172.21.15.107:34046) 2023-07-06T21:39:00.306 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bcb9a700 1 -- 172.21.15.107:0/2619236534 learned_addr learned my addr 172.21.15.107:0/2619236534 (peer_addr_for_me v1:172.21.15.107:0/0) 2023-07-06T21:39:00.307 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bdb9c700 1 -- 172.21.15.107:0/2619236534 >> v1:172.21.15.107:6789/0 conn(0x7f18b80cfd10 legacy=0x7f18b80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bdb9c700 1 -- 172.21.15.107:0/2619236534 >> v1:172.21.15.84:6789/0 conn(0x7f18b80b81c0 legacy=0x7f18b80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18bdb9c700 1 -- 172.21.15.107:0/2619236534 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80d1550 con 0x7f18b80b1a80 2023-07-06T21:39:00.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.304+0000 7f18affff700 1 -- 172.21.15.107:0/2619236534 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266728349 0 0) 0x7f18b80cf330 con 0x7f18b80cfd10 2023-07-06T21:39:00.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.305+0000 7f18bdb9c700 1 --2- 172.21.15.107:0/2619236534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b1a80 0x7f18b80d2a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009d90 tx=0x7f18b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c828a60b28bafdfb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:00.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.305+0000 7f18affff700 1 -- 172.21.15.107:0/2619236534 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b000e070 con 0x7f18b80b1a80 2023-07-06T21:39:00.308 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.305+0000 7f18affff700 1 -- 172.21.15.107:0/2619236534 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b000bbd0 con 0x7f18b80b1a80 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.305+0000 7f18affff700 1 -- 172.21.15.107:0/2619236534 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0012430 con 0x7f18b80b1a80 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 -- 172.21.15.107:0/2619236534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b1a80 msgr2=0x7f18b80d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 --2- 172.21.15.107:0/2619236534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b1a80 0x7f18b80d2a90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f18b0009d90 tx=0x7f18b000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 -- 172.21.15.107:0/2619236534 shutdown_connections 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 --2- 172.21.15.107:0/2619236534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b1a80 0x7f18b80d2a90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 -- 172.21.15.107:0/2619236534 >> 172.21.15.107:0/2619236534 conn(0x7f18b806b2b0 msgr2=0x7f18b806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 -- 172.21.15.107:0/2619236534 shutdown_connections 2023-07-06T21:39:00.309 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.306+0000 7f18bf5ff700 1 -- 172.21.15.107:0/2619236534 wait complete. 2023-07-06T21:39:00.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.307+0000 7f18bf5ff700 1 Processor -- start 2023-07-06T21:39:00.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.308+0000 7f18bf5ff700 1 -- start start 2023-07-06T21:39:00.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bf5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18b80b1a80 0x7f18b8156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bf5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 0x7f18b8141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.311 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bf5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b80cfd10 0x7f18b8141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bf5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8157a10 con 0x7f18b80cfd10 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bf5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8157cf0 con 0x7f18b80b81c0 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bf5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8157fd0 con 0x7f18b80b1a80 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bcb9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 0x7f18b8141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bd39b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18b80b1a80 0x7f18b8156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bcb9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 0x7f18b8141930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46760/0 (socket says 172.21.15.107:46760) 2023-07-06T21:39:00.312 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bcb9a700 1 -- 172.21.15.107:0/189809532 learned_addr learned my addr 172.21.15.107:0/189809532 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bdb9c700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b80cfd10 0x7f18b8141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bd39b700 1 -- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 msgr2=0x7f18b8141930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bd39b700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 0x7f18b8141930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bd39b700 1 -- 172.21.15.107:0/189809532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b80cfd10 msgr2=0x7f18b8141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bd39b700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b80cfd10 0x7f18b8141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bd39b700 1 -- 172.21.15.107:0/189809532 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b0009a70 con 0x7f18b80b1a80 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.309+0000 7f18bcb9a700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 0x7f18b8141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:00.313 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18bd39b700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18b80b1a80 0x7f18b8156f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18b4000f60 tx=0x7f18b400c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:00.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b40052d0 con 0x7f18b80b1a80 2023-07-06T21:39:00.314 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b4005430 con 0x7f18b80b1a80 2023-07-06T21:39:00.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b81582b0 con 0x7f18b80b1a80 2023-07-06T21:39:00.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b40038c0 con 0x7f18b80b1a80 2023-07-06T21:39:00.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81f2020 con 0x7f18b80b1a80 2023-07-06T21:39:00.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f18b4003d20 con 0x7f18b80b1a80 2023-07-06T21:39:00.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.310+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f18b4011d50 con 0x7f18b80b1a80 2023-07-06T21:39:00.315 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.311+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b81587d0 con 0x7f18b80b1a80 2023-07-06T21:39:00.316 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.314+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f18b81587d0 con 0x7f18b80b1a80 2023-07-06T21:39:00.372 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:00.368+0000 7f1cd3851200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T21:39:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.401+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "42a1f42a-4cec-4d2f-8943-efd17495b1c1", "id": 1} v 0) v1 -- 0x7f18b804e1d0 con 0x7f18b80b1a80 2023-07-06T21:39:00.411 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.409+0000 7f18adffb700 1 -- 172.21.15.107:0/189809532 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42a1f42a-4cec-4d2f-8943-efd17495b1c1", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f18b4017070 con 0x7f18b80b1a80 2023-07-06T21:39:00.411 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-06T21:39:00.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18b80b1a80 msgr2=0x7f18b8156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18b80b1a80 0x7f18b8156f50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f18b4000f60 tx=0x7f18b400c730 comp rx=0 tx=0).stop 2023-07-06T21:39:00.412 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 shutdown_connections 2023-07-06T21:39:00.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b80cfd10 0x7f18b8141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18b80b1a80 0x7f18b8156f50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 --2- 172.21.15.107:0/189809532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18b80b81c0 0x7f18b8141930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.410+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 >> 172.21.15.107:0/189809532 conn(0x7f18b806b2b0 msgr2=0x7f18b80d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:00.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.411+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 shutdown_connections 2023-07-06T21:39:00.413 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.411+0000 7f18bf5ff700 1 -- 172.21.15.107:0/189809532 wait complete. 2023-07-06T21:39:00.420 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:00.418+0000 7f2aaff32200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-06T21:39:00.429 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new e354fea8-9ad5-42e4-a409-658253c8b4e4 2 2023-07-06T21:39:00.478 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:00.477+0000 7f1cd3851200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T21:39:00.534 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:00.532+0000 7f2aaff32200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-06T21:39:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.616+0000 7f42afdff700 1 Processor -- start 2023-07-06T21:39:00.618 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.616+0000 7f42afdff700 1 -- start start 2023-07-06T21:39:00.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.617+0000 7f42afdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.617+0000 7f42afdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f42a80ceda0 0x7f42a80cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.617+0000 7f42afdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a80d1830 con 0x7f42a80b1a80 2023-07-06T21:39:00.619 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.617+0000 7f42afdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a80cf330 con 0x7f42a80ceda0 2023-07-06T21:39:00.620 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.617+0000 7f42afdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80d1270 con 0x7f42a80b81c0 2023-07-06T21:39:00.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41788/0 (socket says 172.21.15.107:41788) 2023-07-06T21:39:00.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 -- 172.21.15.107:0/1760999818 learned_addr learned my addr 172.21.15.107:0/1760999818 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:00.621 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42ae39c700 1 --2- 172.21.15.107:0/1760999818 >> v2:172.21.15.107:3300/0 conn(0x7f42a80ceda0 0x7f42a80cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 -- 172.21.15.107:0/1760999818 >> v2:172.21.15.107:3300/0 conn(0x7f42a80ceda0 msgr2=0x7f42a80cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 --2- 172.21.15.107:0/1760999818 >> v2:172.21.15.107:3300/0 conn(0x7f42a80ceda0 0x7f42a80cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 -- 172.21.15.107:0/1760999818 >> v1:172.21.15.107:6789/0 conn(0x7f42a80b1a80 legacy=0x7f42a815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.618+0000 7f42adb9b700 1 -- 172.21.15.107:0/1760999818 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80d1550 con 0x7f42a80b81c0 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.619+0000 7f42adb9b700 1 --2- 172.21.15.107:0/1760999818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80cf600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42a40098f0 tx=0x7f42a4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3291820e939dbadc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.619+0000 7f42acb99700 1 -- 172.21.15.107:0/1760999818 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4004230 con 0x7f42a80b81c0 2023-07-06T21:39:00.622 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.619+0000 7f42acb99700 1 -- 172.21.15.107:0/1760999818 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4004390 con 0x7f42a80b81c0 2023-07-06T21:39:00.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.619+0000 7f42acb99700 1 -- 172.21.15.107:0/1760999818 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40036a0 con 0x7f42a80b81c0 2023-07-06T21:39:00.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.619+0000 7f42afdff700 1 -- 172.21.15.107:0/1760999818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 msgr2=0x7f42a80cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.623 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.619+0000 7f42afdff700 1 --2- 172.21.15.107:0/1760999818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80cf600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42a40098f0 tx=0x7f42a4009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:00.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.620+0000 7f42afdff700 1 -- 172.21.15.107:0/1760999818 shutdown_connections 2023-07-06T21:39:00.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.620+0000 7f42afdff700 1 --2- 172.21.15.107:0/1760999818 >> v2:172.21.15.107:3300/0 conn(0x7f42a80ceda0 0x7f42a80cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.620+0000 7f42afdff700 1 --2- 172.21.15.107:0/1760999818 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80cf600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.620+0000 7f42afdff700 1 -- 172.21.15.107:0/1760999818 >> 172.21.15.107:0/1760999818 conn(0x7f42a806b2b0 msgr2=0x7f42a806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:00.624 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.620+0000 7f42afdff700 1 -- 172.21.15.107:0/1760999818 shutdown_connections 2023-07-06T21:39:00.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.620+0000 7f42afdff700 1 -- 172.21.15.107:0/1760999818 wait complete. 2023-07-06T21:39:00.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.622+0000 7f42afdff700 1 Processor -- start 2023-07-06T21:39:00.625 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.622+0000 7f42afdff700 1 -- start start 2023-07-06T21:39:00.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.623+0000 7f42afdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42a80b1a80 0x7f42a80c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42afdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42afdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 0x7f42a80b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.627 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42afdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80c7a80 con 0x7f42a80b1a80 2023-07-06T21:39:00.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42afdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80c6e00 con 0x7f42a80ceda0 2023-07-06T21:39:00.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42afdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80c70e0 con 0x7f42a80b81c0 2023-07-06T21:39:00.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ae39c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 0x7f42a80b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.628 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ae39c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 0x7f42a80b3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46778/0 (socket says 172.21.15.107:46778) 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ae39c700 1 -- 172.21.15.107:0/2859702928 learned_addr learned my addr 172.21.15.107:0/2859702928 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41794/0 (socket says 172.21.15.107:41794) 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42adb9b700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42a80b1a80 0x7f42a80c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 -- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 msgr2=0x7f42a80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 0x7f42a80b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 -- 172.21.15.107:0/2859702928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42a80b1a80 msgr2=0x7f42a80c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.629 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42a80b1a80 0x7f42a80c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ad39a700 1 -- 172.21.15.107:0/2859702928 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a4009550 con 0x7f42a80b81c0 2023-07-06T21:39:00.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.624+0000 7f42ae39c700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 0x7f42a80b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:00.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f42ad39a700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80c7ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f429800c400 tx=0x7f429800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:00.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298011790 con 0x7f42a80b81c0 2023-07-06T21:39:00.630 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42980118f0 con 0x7f42a80b81c0 2023-07-06T21:39:00.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298011ca0 con 0x7f42a80b81c0 2023-07-06T21:39:00.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a80c73c0 con 0x7f42a80b81c0 2023-07-06T21:39:00.631 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80c83e0 con 0x7f42a80b81c0 2023-07-06T21:39:00.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f429801c760 con 0x7f42a80b81c0 2023-07-06T21:39:00.632 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.625+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f42980128e0 con 0x7f42a80b81c0 2023-07-06T21:39:00.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.626+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80c85a0 con 0x7f42a80b81c0 2023-07-06T21:39:00.633 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.628+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f42a80c85a0 con 0x7f42a80b81c0 2023-07-06T21:39:00.717 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.715+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e354fea8-9ad5-42e4-a409-658253c8b4e4", "id": 2} v 0) v1 -- 0x7f42a80c7bc0 con 0x7f42a80b81c0 2023-07-06T21:39:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.723+0000 7f429e7fc700 1 -- 172.21.15.107:0/2859702928 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e354fea8-9ad5-42e4-a409-658253c8b4e4", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f42a80c7bc0 con 0x7f42a80b81c0 2023-07-06T21:39:00.726 INFO:teuthology.orchestra.run.smithi107.stdout:2 2023-07-06T21:39:00.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.724+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 msgr2=0x7f42a80c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.727 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.724+0000 7f42afdff700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80c7ea0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f429800c400 tx=0x7f429800c710 comp rx=0 tx=0).stop 2023-07-06T21:39:00.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.724+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 shutdown_connections 2023-07-06T21:39:00.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.724+0000 7f42afdff700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42a80ceda0 0x7f42a80b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.724+0000 7f42afdff700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42a80b1a80 0x7f42a80c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.728 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.724+0000 7f42afdff700 1 --2- 172.21.15.107:0/2859702928 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42a80b81c0 0x7f42a80c7ea0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.725+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 >> 172.21.15.107:0/2859702928 conn(0x7f42a806b2b0 msgr2=0x7f42a80b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:00.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.725+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 shutdown_connections 2023-07-06T21:39:00.729 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.725+0000 7f42afdff700 1 -- 172.21.15.107:0/2859702928 wait complete. 2023-07-06T21:39:00.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new a531fbd4-2b94-4043-b9dd-6c6c106f5321 3 2023-07-06T21:39:00.941 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.939+0000 7f234d9ff700 1 Processor -- start 2023-07-06T21:39:00.941 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.939+0000 7f234d9ff700 1 -- start start 2023-07-06T21:39:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f234d9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f23480b81c0 0x7f23480cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f234d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f23480cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f234d9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23480d1830 con 0x7f23480b1a80 2023-07-06T21:39:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f234d9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23480cf330 con 0x7f23480b81c0 2023-07-06T21:39:00.944 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f234d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480d1270 con 0x7f23480ceda0 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f23477fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f23480cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f23477fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f23480cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41808/0 (socket says 172.21.15.107:41808) 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f23477fe700 1 -- 172.21.15.107:0/3670785986 learned_addr learned my addr 172.21.15.107:0/3670785986 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f2346ffd700 1 --2- 172.21.15.107:0/3670785986 >> v2:172.21.15.84:3300/0 conn(0x7f23480b81c0 0x7f23480cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.941+0000 7f233ffff700 1 --1- 172.21.15.107:0/3670785986 >> v1:172.21.15.107:6789/0 conn(0x7f23480b1a80 0x7f234815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:34064/0 (socket says 172.21.15.107:34064) 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23477fe700 1 -- 172.21.15.107:0/3670785986 >> v2:172.21.15.84:3300/0 conn(0x7f23480b81c0 msgr2=0x7f23480cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23477fe700 1 --2- 172.21.15.107:0/3670785986 >> v2:172.21.15.84:3300/0 conn(0x7f23480b81c0 0x7f23480cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.945 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23477fe700 1 -- 172.21.15.107:0/3670785986 >> v1:172.21.15.107:6789/0 conn(0x7f23480b1a80 legacy=0x7f234815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23477fe700 1 -- 172.21.15.107:0/3670785986 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23480d1550 con 0x7f23480ceda0 2023-07-06T21:39:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23477fe700 1 --2- 172.21.15.107:0/3670785986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f23480cfb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2340009e50 tx=0x7f234000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db67c69d86a57392 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23467fc700 1 -- 172.21.15.107:0/3670785986 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000e070 con 0x7f23480ceda0 2023-07-06T21:39:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23467fc700 1 -- 172.21.15.107:0/3670785986 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000b910 con 0x7f23480ceda0 2023-07-06T21:39:00.946 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.942+0000 7f23467fc700 1 -- 172.21.15.107:0/3670785986 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000bcc0 con 0x7f23480ceda0 2023-07-06T21:39:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 -- 172.21.15.107:0/3670785986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 msgr2=0x7f23480cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.947 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 --2- 172.21.15.107:0/3670785986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f23480cfb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2340009e50 tx=0x7f234000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:00.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 -- 172.21.15.107:0/3670785986 shutdown_connections 2023-07-06T21:39:00.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 --2- 172.21.15.107:0/3670785986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f23480cfb40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 --2- 172.21.15.107:0/3670785986 >> v2:172.21.15.84:3300/0 conn(0x7f23480b81c0 0x7f23480cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 -- 172.21.15.107:0/3670785986 >> 172.21.15.107:0/3670785986 conn(0x7f234806b2b0 msgr2=0x7f234806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:00.948 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 -- 172.21.15.107:0/3670785986 shutdown_connections 2023-07-06T21:39:00.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.943+0000 7f234d9ff700 1 -- 172.21.15.107:0/3670785986 wait complete. 2023-07-06T21:39:00.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.945+0000 7f234d9ff700 1 Processor -- start 2023-07-06T21:39:00.949 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.945+0000 7f234d9ff700 1 -- start start 2023-07-06T21:39:00.950 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.946+0000 7f234d9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23480b1a80 0x7f2348156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.946+0000 7f234d9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23480b81c0 0x7f23480b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.946+0000 7f234d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f2348152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.946+0000 7f234d9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481e6c80 con 0x7f23480b1a80 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.946+0000 7f234d9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481e6f20 con 0x7f23480b81c0 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.946+0000 7f234d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23481e7200 con 0x7f23480ceda0 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f233ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23480b81c0 0x7f23480b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.951 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f2348152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f2346ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23480b1a80 0x7f2348156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f2348152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41820/0 (socket says 172.21.15.107:41820) 2023-07-06T21:39:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f2346ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23480b1a80 0x7f2348156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.107:45332/0 (socket says 172.21.15.107:45332) 2023-07-06T21:39:00.952 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 -- 172.21.15.107:0/1274056662 learned_addr learned my addr 172.21.15.107:0/1274056662 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 -- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23480b81c0 msgr2=0x7f23480b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23480b81c0 0x7f23480b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 -- 172.21.15.107:0/1274056662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23480b1a80 msgr2=0x7f2348156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23480b1a80 0x7f2348156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 -- 172.21.15.107:0/1274056662 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009b30 con 0x7f23480ceda0 2023-07-06T21:39:00.953 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f23477fe700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f2348152a60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23480d13b0 tx=0x7f2340004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2340004800 con 0x7f23480ceda0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.947+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2340004960 con 0x7f23480ceda0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.948+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234001c680 con 0x7f23480ceda0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.948+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23481e74e0 con 0x7f23480ceda0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.948+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23480b42f0 con 0x7f23480ceda0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.948+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f234001cae0 con 0x7f23480ceda0 2023-07-06T21:39:00.954 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.948+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f2340025ce0 con 0x7f23480ceda0 2023-07-06T21:39:00.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.949+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23480b4480 con 0x7f23480ceda0 2023-07-06T21:39:00.955 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:00.953+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2340044b00 con 0x7f23480ceda0 2023-07-06T21:39:01.040 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.038+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a531fbd4-2b94-4043-b9dd-6c6c106f5321", "id": 3} v 0) v1 -- 0x7f234804e1d0 con 0x7f23480ceda0 2023-07-06T21:39:01.049 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.047+0000 7f2344ff9700 1 -- 172.21.15.107:0/1274056662 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a531fbd4-2b94-4043-b9dd-6c6c106f5321", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2340033030 con 0x7f23480ceda0 2023-07-06T21:39:01.049 INFO:teuthology.orchestra.run.smithi107.stdout:3 2023-07-06T21:39:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 msgr2=0x7f2348152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f2348152a60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f23480d13b0 tx=0x7f2340004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 shutdown_connections 2023-07-06T21:39:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23480ceda0 0x7f2348152a60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.050 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23480b1a80 0x7f2348156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 --2- 172.21.15.107:0/1274056662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23480b81c0 0x7f23480b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 >> 172.21.15.107:0/1274056662 conn(0x7f234806b2b0 msgr2=0x7f23480d22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 shutdown_connections 2023-07-06T21:39:01.051 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.048+0000 7f234d9ff700 1 -- 172.21.15.107:0/1274056662 wait complete. 2023-07-06T21:39:01.059 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:01.057+0000 7f2aaff32200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T21:39:01.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 4c8b22f3-ae3f-42af-80b7-63029b5f82f0 4 2023-07-06T21:39:01.071 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:01.067+0000 7f1cd3851200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-06T21:39:01.256 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.255+0000 7fb00edff700 1 Processor -- start 2023-07-06T21:39:01.257 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.255+0000 7fb00edff700 1 -- start start 2023-07-06T21:39:01.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00edff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 0x7fb0080cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00edff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb00806c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00edff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0081600b0 con 0x7fb00806d7d0 2023-07-06T21:39:01.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00edff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb008160950 con 0x7fb00815c700 2023-07-06T21:39:01.258 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00edff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080cabc0 con 0x7fb00806a5a0 2023-07-06T21:39:01.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00cb9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 0x7fb0080cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00d39c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb00806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00cb9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 0x7fb0080cc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46792/0 (socket says 172.21.15.107:46792) 2023-07-06T21:39:01.259 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.256+0000 7fb00cb9b700 1 -- 172.21.15.107:0/3033201944 learned_addr learned my addr 172.21.15.107:0/3033201944 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:01.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb00d39c700 1 -- 172.21.15.107:0/3033201944 >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 msgr2=0x7fb0080cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:01.260 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb00d39c700 1 --2- 172.21.15.107:0/3033201944 >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 0x7fb0080cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb00d39c700 1 -- 172.21.15.107:0/3033201944 >> v1:172.21.15.84:6789/0 conn(0x7fb00806d7d0 legacy=0x7fb008069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb00d39c700 1 -- 172.21.15.107:0/3033201944 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0080cca40 con 0x7fb00806a5a0 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb00cb9b700 1 --2- 172.21.15.107:0/3033201944 >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 0x7fb0080cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb00d39c700 1 --2- 172.21.15.107:0/3033201944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb00806c400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb000009d90 tx=0x7fb00000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a161553b74d373a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb0077fe700 1 -- 172.21.15.107:0/3033201944 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00000e070 con 0x7fb00806a5a0 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb0077fe700 1 -- 172.21.15.107:0/3033201944 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00000bb20 con 0x7fb00806a5a0 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.257+0000 7fb0077fe700 1 -- 172.21.15.107:0/3033201944 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000012430 con 0x7fb00806a5a0 2023-07-06T21:39:01.261 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 -- 172.21.15.107:0/3033201944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 msgr2=0x7fb00806c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 --2- 172.21.15.107:0/3033201944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb00806c400 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb000009d90 tx=0x7fb00000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 -- 172.21.15.107:0/3033201944 shutdown_connections 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 --2- 172.21.15.107:0/3033201944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb00806c400 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 --2- 172.21.15.107:0/3033201944 >> v2:172.21.15.107:3300/0 conn(0x7fb00815c700 0x7fb0080cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 -- 172.21.15.107:0/3033201944 >> 172.21.15.107:0/3033201944 conn(0x7fb0080cbf10 msgr2=0x7fb0080ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 -- 172.21.15.107:0/3033201944 shutdown_connections 2023-07-06T21:39:01.262 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.258+0000 7fb00edff700 1 -- 172.21.15.107:0/3033201944 wait complete. 2023-07-06T21:39:01.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.260+0000 7fb00edff700 1 Processor -- start 2023-07-06T21:39:01.263 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.261+0000 7fb00edff700 1 -- start start 2023-07-06T21:39:01.264 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00edff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb008159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00edff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00806d7d0 0x7fb0080b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00edff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb00815c700 0x7fb0080b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00edff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080c5a70 con 0x7fb00806d7d0 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00edff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00815a4b0 con 0x7fb00815c700 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00edff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00815f030 con 0x7fb00806a5a0 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb008159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00d39c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb00815c700 0x7fb0080b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.265 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb007fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00806d7d0 0x7fb0080b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb008159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41828/0 (socket says 172.21.15.107:41828) 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 -- 172.21.15.107:0/2915556083 learned_addr learned my addr 172.21.15.107:0/2915556083 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 -- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb00815c700 msgr2=0x7fb0080b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb00815c700 0x7fb0080b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 -- 172.21.15.107:0/2915556083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00806d7d0 msgr2=0x7fb0080b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00806d7d0 0x7fb0080b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.266 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.262+0000 7fb00cb9b700 1 -- 172.21.15.107:0/2915556083 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc009550 con 0x7fb00806a5a0 2023-07-06T21:39:01.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb00cb9b700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb008159f70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faffc00ec10 tx=0x7faffc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:01.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00c850 con 0x7fb00806a5a0 2023-07-06T21:39:01.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc00c9b0 con 0x7fb00806a5a0 2023-07-06T21:39:01.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc00cd40 con 0x7fb00806a5a0 2023-07-06T21:39:01.267 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb000009a70 con 0x7fb00806a5a0 2023-07-06T21:39:01.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00815f5f0 con 0x7fb00806a5a0 2023-07-06T21:39:01.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faffc013770 con 0x7fb00806a5a0 2023-07-06T21:39:01.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.263+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7faffc01c720 con 0x7fb00806a5a0 2023-07-06T21:39:01.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.264+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00804e1d0 con 0x7fb00806a5a0 2023-07-06T21:39:01.268 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.267+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7faffc021060 con 0x7fb00806a5a0 2023-07-06T21:39:01.352 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.350+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4c8b22f3-ae3f-42af-80b7-63029b5f82f0", "id": 4} v 0) v1 -- 0x7fb00806a350 con 0x7fb00806a5a0 2023-07-06T21:39:01.361 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.359+0000 7fb0057fa700 1 -- 172.21.15.107:0/2915556083 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4c8b22f3-ae3f-42af-80b7-63029b5f82f0", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7faffc02a020 con 0x7fb00806a5a0 2023-07-06T21:39:01.361 INFO:teuthology.orchestra.run.smithi107.stdout:4 2023-07-06T21:39:01.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 msgr2=0x7fb008159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.362 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb008159f70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faffc00ec10 tx=0x7faffc00ef20 comp rx=0 tx=0).stop 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 shutdown_connections 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb00815c700 0x7fb0080b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb00806a5a0 0x7fb008159f70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 --2- 172.21.15.107:0/2915556083 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb00806d7d0 0x7fb0080b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 >> 172.21.15.107:0/2915556083 conn(0x7fb0080cbf10 msgr2=0x7fb00815bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 shutdown_connections 2023-07-06T21:39:01.363 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.360+0000 7fb00edff700 1 -- 172.21.15.107:0/2915556083 wait complete. 2023-07-06T21:39:01.379 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new a66f2d6d-f11e-4f56-a186-44033dc51676 5 2023-07-06T21:39:01.580 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.577+0000 7fefe29ff700 1 Processor -- start 2023-07-06T21:39:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.578+0000 7fefe29ff700 1 -- start start 2023-07-06T21:39:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefe29ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fefdc070db0 0x7fefdc15c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefe29ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0ae0c0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefe29ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefdc0cc9e0 con 0x7fefdc069e90 2023-07-06T21:39:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefe29ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc0cccc0 con 0x7fefdc070db0 2023-07-06T21:39:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefdb7fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fefdc070db0 0x7fefdc15c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefdb7fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fefdc070db0 0x7fefdc15c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46804/0 (socket says 172.21.15.107:46804) 2023-07-06T21:39:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.580+0000 7fefdb7fe700 1 -- 172.21.15.107:0/817067653 learned_addr learned my addr 172.21.15.107:0/817067653 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.581+0000 7fefdb7fe700 1 -- 172.21.15.107:0/817067653 >> v1:172.21.15.107:6789/0 conn(0x7fefdc069e90 legacy=0x7fefdc06a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.581+0000 7fefdb7fe700 1 -- 172.21.15.107:0/817067653 >> v1:172.21.15.84:6789/0 conn(0x7fefdc0c8e50 legacy=0x7fefdc0c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.581+0000 7fefdb7fe700 1 -- 172.21.15.107:0/817067653 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc0ccfa0 con 0x7fefdc070db0 2023-07-06T21:39:01.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.581+0000 7fefdb7fe700 1 --2- 172.21.15.107:0/817067653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc070db0 0x7fefdc15c910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fefd00091e0 tx=0x7fefd00094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ed317bdee31e17 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:01.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.582+0000 7fefdaffd700 1 -- 172.21.15.107:0/817067653 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0004230 con 0x7fefdc070db0 2023-07-06T21:39:01.585 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.582+0000 7fefdaffd700 1 -- 172.21.15.107:0/817067653 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0004390 con 0x7fefdc070db0 2023-07-06T21:39:01.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.582+0000 7fefdaffd700 1 -- 172.21.15.107:0/817067653 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000f740 con 0x7fefdc070db0 2023-07-06T21:39:01.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 -- 172.21.15.107:0/817067653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc070db0 msgr2=0x7fefdc15c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 --2- 172.21.15.107:0/817067653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc070db0 0x7fefdc15c910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fefd00091e0 tx=0x7fefd00094f0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.586 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 -- 172.21.15.107:0/817067653 shutdown_connections 2023-07-06T21:39:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 --2- 172.21.15.107:0/817067653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc070db0 0x7fefdc15c910 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 -- 172.21.15.107:0/817067653 >> 172.21.15.107:0/817067653 conn(0x7fefdc0cbed0 msgr2=0x7fefdc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 -- 172.21.15.107:0/817067653 shutdown_connections 2023-07-06T21:39:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.583+0000 7fefe29ff700 1 -- 172.21.15.107:0/817067653 wait complete. 2023-07-06T21:39:01.588 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.586+0000 7fefe29ff700 1 Processor -- start 2023-07-06T21:39:01.589 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.586+0000 7fefe29ff700 1 -- start start 2023-07-06T21:39:01.590 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe29ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc069e90 0x7fefdc0ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe29ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefdc070db0 0x7fefdc1518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe29ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 0x7fefdc1510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe29ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc152290 con 0x7fefdc0c8e50 2023-07-06T21:39:01.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe29ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0ad430 con 0x7fefdc069e90 2023-07-06T21:39:01.591 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe29ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0acb90 con 0x7fefdc070db0 2023-07-06T21:39:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe0f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 0x7fefdc1510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe0f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 0x7fefdc1510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.107:45348/0 (socket says 172.21.15.107:45348) 2023-07-06T21:39:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefe0f9c700 1 -- 172.21.15.107:0/3444174467 learned_addr learned my addr 172.21.15.107:0/3444174467 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefdbfff700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc069e90 0x7fefdc0ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.588+0000 7fefdb7fe700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefdc070db0 0x7fefdc1518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.592 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.589+0000 7fefe0f9c700 1 -- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefdc070db0 msgr2=0x7fefdc1518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.589+0000 7fefe0f9c700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefdc070db0 0x7fefdc1518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.589+0000 7fefe0f9c700 1 -- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc069e90 msgr2=0x7fefdc0ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.589+0000 7fefe0f9c700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc069e90 0x7fefdc0ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.589+0000 7fefe0f9c700 1 -- 172.21.15.107:0/3444174467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd0009de0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.593 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.589+0000 7fefe0f9c700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 0x7fefdc1510e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b9c0 tx=0x7fefd400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.590+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd400d8e0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.590+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd400da40 con 0x7fefdc0c8e50 2023-07-06T21:39:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.590+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd400ddf0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.594 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.590+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc0ace70 con 0x7fefdc0c8e50 2023-07-06T21:39:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.590+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc0c59e0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.591+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fefd4006d50 con 0x7fefdc0c8e50 2023-07-06T21:39:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.591+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc04e1d0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.595 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.591+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fefd4011070 con 0x7fefdc0c8e50 2023-07-06T21:39:01.598 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.596+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fefd4023060 con 0x7fefdc0c8e50 2023-07-06T21:39:01.699 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.696+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a66f2d6d-f11e-4f56-a186-44033dc51676", "id": 5} v 0) v1 -- 0x7fefdc15c0e0 con 0x7fefdc0c8e50 2023-07-06T21:39:01.706 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.704+0000 7fefd8ff9700 1 -- 172.21.15.107:0/3444174467 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a66f2d6d-f11e-4f56-a186-44033dc51676", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fefd4015420 con 0x7fefdc0c8e50 2023-07-06T21:39:01.707 INFO:teuthology.orchestra.run.smithi107.stdout:5 2023-07-06T21:39:01.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 msgr2=0x7fefdc1510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 0x7fefdc1510e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fefd400b9c0 tx=0x7fefd400bcd0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.708 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 shutdown_connections 2023-07-06T21:39:01.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fefdc0c8e50 0x7fefdc1510e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefdc069e90 0x7fefdc0ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 --2- 172.21.15.107:0/3444174467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefdc070db0 0x7fefdc1518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 >> 172.21.15.107:0/3444174467 conn(0x7fefdc0cbed0 msgr2=0x7fefdc157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:01.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 shutdown_connections 2023-07-06T21:39:01.709 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.705+0000 7fefe29ff700 1 -- 172.21.15.107:0/3444174467 wait complete. 2023-07-06T21:39:01.732 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new c5fc70b9-b03f-4319-8d63-f793ebd4e6ae 6 2023-07-06T21:39:01.752 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:01.750+0000 7f2aaff32200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T21:39:01.789 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:01.787+0000 7f1cd3851200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-06T21:39:01.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.921+0000 7fbf6cdff700 1 Processor -- start 2023-07-06T21:39:01.924 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.921+0000 7fbf6cdff700 1 -- start start 2023-07-06T21:39:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf6cdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbf680cfd10 0x7fbf680d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf6cdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 0x7fbf680d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf6cdff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680d1830 con 0x7fbf680b81c0 2023-07-06T21:39:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf6cdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf680cf330 con 0x7fbf680cfd10 2023-07-06T21:39:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf6cdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf680d1270 con 0x7fbf680b1a80 2023-07-06T21:39:01.926 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf65d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbf680cfd10 0x7fbf680d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf66d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 0x7fbf680d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf65d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbf680cfd10 0x7fbf680d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.107:45352/0 (socket says 172.21.15.107:45352) 2023-07-06T21:39:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf66d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 0x7fbf680d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46818/0 (socket says 172.21.15.107:46818) 2023-07-06T21:39:01.927 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.923+0000 7fbf65d9b700 1 -- 172.21.15.107:0/4152974609 learned_addr learned my addr 172.21.15.107:0/4152974609 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf65d9b700 1 -- 172.21.15.107:0/4152974609 >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 msgr2=0x7fbf680d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf65d9b700 1 --2- 172.21.15.107:0/4152974609 >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 0x7fbf680d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf65d9b700 1 -- 172.21.15.107:0/4152974609 >> v1:172.21.15.84:6789/0 conn(0x7fbf680b81c0 legacy=0x7fbf680cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf65d9b700 1 -- 172.21.15.107:0/4152974609 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680d1550 con 0x7fbf680cfd10 2023-07-06T21:39:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf66d9d700 1 --2- 172.21.15.107:0/4152974609 >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 0x7fbf680d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:01.929 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf6559a700 1 -- 172.21.15.107:0/4152974609 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324015262 0 0) 0x7fbf680d1830 con 0x7fbf680b81c0 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf65d9b700 1 --2- 172.21.15.107:0/4152974609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680cfd10 0x7fbf680d2380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf680b8d30 tx=0x7fbf5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d3ed24ab9ef25d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf6559a700 1 -- 172.21.15.107:0/4152974609 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011830 con 0x7fbf680cfd10 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf6559a700 1 -- 172.21.15.107:0/4152974609 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c011990 con 0x7fbf680cfd10 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.924+0000 7fbf6559a700 1 -- 172.21.15.107:0/4152974609 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c011d20 con 0x7fbf680cfd10 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 -- 172.21.15.107:0/4152974609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680cfd10 msgr2=0x7fbf680d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/4152974609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680cfd10 0x7fbf680d2380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbf680b8d30 tx=0x7fbf5c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 -- 172.21.15.107:0/4152974609 shutdown_connections 2023-07-06T21:39:01.930 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/4152974609 >> v2:172.21.15.107:3300/0 conn(0x7fbf680b1a80 0x7fbf680d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/4152974609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680cfd10 0x7fbf680d2380 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 -- 172.21.15.107:0/4152974609 >> 172.21.15.107:0/4152974609 conn(0x7fbf6806b2b0 msgr2=0x7fbf6806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 -- 172.21.15.107:0/4152974609 shutdown_connections 2023-07-06T21:39:01.931 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.925+0000 7fbf6cdff700 1 -- 172.21.15.107:0/4152974609 wait complete. 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.927+0000 7fbf6cdff700 1 Processor -- start 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.927+0000 7fbf6cdff700 1 -- start start 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6cdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680b1a80 0x7fbf6815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6cdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 0x7fbf680c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf680cfd10 0x7fbf681f24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6cdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf681f2090 con 0x7fbf680b1a80 2023-07-06T21:39:01.933 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6cdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680c67c0 con 0x7fbf680b81c0 2023-07-06T21:39:01.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680c6d10 con 0x7fbf680cfd10 2023-07-06T21:39:01.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 0x7fbf680c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf66d9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf680cfd10 0x7fbf681f24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 0x7fbf680c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46832/0 (socket says 172.21.15.107:46832) 2023-07-06T21:39:01.934 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 -- 172.21.15.107:0/2554600585 learned_addr learned my addr 172.21.15.107:0/2554600585 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf6659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680b1a80 0x7fbf6815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 -- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf680cfd10 msgr2=0x7fbf681f24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf680cfd10 0x7fbf681f24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 -- 172.21.15.107:0/2554600585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680b1a80 msgr2=0x7fbf6815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680b1a80 0x7fbf6815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf65d9b700 1 -- 172.21.15.107:0/2554600585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf60009a70 con 0x7fbf680b81c0 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.928+0000 7fbf66d9d700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf680cfd10 0x7fbf681f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:01.935 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf65d9b700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 0x7fbf680c70b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009c10 tx=0x7fbf5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:01.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c004550 con 0x7fbf680b81c0 2023-07-06T21:39:01.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c0036a0 con 0x7fbf680b81c0 2023-07-06T21:39:01.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c009550 con 0x7fbf680b81c0 2023-07-06T21:39:01.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003a10 con 0x7fbf680b81c0 2023-07-06T21:39:01.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf681f1cb0 con 0x7fbf680b81c0 2023-07-06T21:39:01.936 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbf5c01a430 con 0x7fbf680b81c0 2023-07-06T21:39:01.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.929+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fbf5c021030 con 0x7fbf680b81c0 2023-07-06T21:39:01.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.930+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf681f21d0 con 0x7fbf680b81c0 2023-07-06T21:39:01.937 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:01.933+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbf681f21d0 con 0x7fbf680b81c0 2023-07-06T21:39:01.938 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:01.936+0000 7f2aaff32200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T21:39:01.983 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:01.981+0000 7f1cd3851200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-06T21:39:02.023 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.020+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c5fc70b9-b03f-4319-8d63-f793ebd4e6ae", "id": 6} v 0) v1 -- 0x7fbf6804e1d0 con 0x7fbf680b81c0 2023-07-06T21:39:02.031 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.028+0000 7fbf56ffd700 1 -- 172.21.15.107:0/2554600585 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c5fc70b9-b03f-4319-8d63-f793ebd4e6ae", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbf5c02b060 con 0x7fbf680b81c0 2023-07-06T21:39:02.032 INFO:teuthology.orchestra.run.smithi107.stdout:6 2023-07-06T21:39:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.029+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 msgr2=0x7fbf680c70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.029+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 0x7fbf680c70b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009c10 tx=0x7fbf5c004750 comp rx=0 tx=0).stop 2023-07-06T21:39:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 shutdown_connections 2023-07-06T21:39:02.033 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf680cfd10 0x7fbf681f24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf680b1a80 0x7fbf6815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 --2- 172.21.15.107:0/2554600585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf680b81c0 0x7fbf680c70b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 >> 172.21.15.107:0/2554600585 conn(0x7fbf6806b2b0 msgr2=0x7fbf680d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 shutdown_connections 2023-07-06T21:39:02.034 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.030+0000 7fbf6cdff700 1 -- 172.21.15.107:0/2554600585 wait complete. 2023-07-06T21:39:02.049 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new ca2f0297-5efb-4620-83a1-8f137a4a0718 7 2023-07-06T21:39:02.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.232+0000 7fc18d5ff700 1 Processor -- start 2023-07-06T21:39:02.235 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.232+0000 7fc18d5ff700 1 -- start start 2023-07-06T21:39:02.236 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.233+0000 7fc18d5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc1880d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.233+0000 7fc18d5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880d1830 con 0x7fc1880b81c0 2023-07-06T21:39:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.233+0000 7fc18d5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1880cf330 con 0x7fc1880cfd10 2023-07-06T21:39:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.233+0000 7fc18d5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880d1270 con 0x7fc1880b1a80 2023-07-06T21:39:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1867fc700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc1880cfd10 0x7fc1880d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:34082/0 (socket says 172.21.15.107:34082) 2023-07-06T21:39:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1877fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc1880d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.237 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1867fc700 1 -- 172.21.15.107:0/2704228679 learned_addr learned my addr 172.21.15.107:0/2704228679 (peer_addr_for_me v1:172.21.15.107:0/0) 2023-07-06T21:39:02.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc185ffb700 1 -- 172.21.15.107:0/2704228679 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2826890436 0 0) 0x7fc1880cf330 con 0x7fc1880cfd10 2023-07-06T21:39:02.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc185ffb700 1 -- 172.21.15.107:0/2704228679 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1700035b0 con 0x7fc1880cfd10 2023-07-06T21:39:02.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1877fe700 1 -- 172.21.15.107:0/2704228679 >> v1:172.21.15.107:6789/0 conn(0x7fc1880cfd10 legacy=0x7fc1880d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1877fe700 1 -- 172.21.15.107:0/2704228679 >> v1:172.21.15.84:6789/0 conn(0x7fc1880b81c0 legacy=0x7fc1880cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1877fe700 1 -- 172.21.15.107:0/2704228679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1880d1550 con 0x7fc1880b1a80 2023-07-06T21:39:02.239 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.234+0000 7fc1877fe700 1 --2- 172.21.15.107:0/2704228679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc1880d2a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc180009dc0 tx=0x7fc18000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3fa0e2ffb8d6fad server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc185ffb700 1 -- 172.21.15.107:0/2704228679 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18000e070 con 0x7fc1880b1a80 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc185ffb700 1 -- 172.21.15.107:0/2704228679 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18000bb20 con 0x7fc1880b1a80 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc185ffb700 1 -- 172.21.15.107:0/2704228679 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180012430 con 0x7fc1880b1a80 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc18d5ff700 1 -- 172.21.15.107:0/2704228679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 msgr2=0x7fc1880d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc18d5ff700 1 --2- 172.21.15.107:0/2704228679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc1880d2a90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc180009dc0 tx=0x7fc18000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc18d5ff700 1 -- 172.21.15.107:0/2704228679 shutdown_connections 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc18d5ff700 1 --2- 172.21.15.107:0/2704228679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc1880d2a90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.240 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.235+0000 7fc18d5ff700 1 -- 172.21.15.107:0/2704228679 >> 172.21.15.107:0/2704228679 conn(0x7fc18806b2b0 msgr2=0x7fc18806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:02.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.236+0000 7fc18d5ff700 1 -- 172.21.15.107:0/2704228679 shutdown_connections 2023-07-06T21:39:02.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.236+0000 7fc18d5ff700 1 -- 172.21.15.107:0/2704228679 wait complete. 2023-07-06T21:39:02.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.238+0000 7fc18d5ff700 1 Processor -- start 2023-07-06T21:39:02.241 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.238+0000 7fc18d5ff700 1 -- start start 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.239+0000 7fc18d5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc188149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.239+0000 7fc18d5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1880b81c0 0x7fc18815ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.239+0000 7fc18d5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1880cfd10 0x7fc1880c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.239+0000 7fc18d5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc188156a80 con 0x7fc1880cfd10 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.239+0000 7fc18d5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880c7bd0 con 0x7fc1880b81c0 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.239+0000 7fc18d5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1880c7eb0 con 0x7fc1880b1a80 2023-07-06T21:39:02.243 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc188149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc1867fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1880b81c0 0x7fc18815ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc188149180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:41866/0 (socket says 172.21.15.107:41866) 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 -- 172.21.15.107:0/1613215817 learned_addr learned my addr 172.21.15.107:0/1613215817 (peer_addr_for_me v2:172.21.15.107:0/0) 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc1877fe700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1880cfd10 0x7fc1880c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 -- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1880b81c0 msgr2=0x7fc18815ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1880b81c0 0x7fc18815ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.244 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 -- 172.21.15.107:0/1613215817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1880cfd10 msgr2=0x7fc1880c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1880cfd10 0x7fc1880c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 -- 172.21.15.107:0/1613215817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180009a70 con 0x7fc1880b1a80 2023-07-06T21:39:02.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc1867fc700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1880b81c0 0x7fc18815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:02.245 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.240+0000 7fc186ffd700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc188149180 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c4e0 tx=0x7fc17800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:02.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780052d0 con 0x7fc1880b1a80 2023-07-06T21:39:02.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178005430 con 0x7fc1880b1a80 2023-07-06T21:39:02.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1880c8600 con 0x7fc1880b1a80 2023-07-06T21:39:02.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780038c0 con 0x7fc1880b1a80 2023-07-06T21:39:02.246 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1880c8190 con 0x7fc1880b1a80 2023-07-06T21:39:02.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc178003d20 con 0x7fc1880b1a80 2023-07-06T21:39:02.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.241+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fc178012a80 con 0x7fc1880b1a80 2023-07-06T21:39:02.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.242+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18815cf60 con 0x7fc1880b1a80 2023-07-06T21:39:02.247 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.244+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fc178021060 con 0x7fc1880b1a80 2023-07-06T21:39:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.330+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ca2f0297-5efb-4620-83a1-8f137a4a0718", "id": 7} v 0) v1 -- 0x7fc18804e1d0 con 0x7fc1880b1a80 2023-07-06T21:39:02.341 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.339+0000 7fc16f7fe700 1 -- 172.21.15.107:0/1613215817 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ca2f0297-5efb-4620-83a1-8f137a4a0718", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc178017070 con 0x7fc1880b1a80 2023-07-06T21:39:02.341 INFO:teuthology.orchestra.run.smithi107.stdout:7 2023-07-06T21:39:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 msgr2=0x7fc188149180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.342 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc188149180 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c4e0 tx=0x7fc17800c7f0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 shutdown_connections 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc1880cfd10 0x7fc1880c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1880b1a80 0x7fc188149180 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 --2- 172.21.15.107:0/1613215817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1880b81c0 0x7fc18815ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 >> 172.21.15.107:0/1613215817 conn(0x7fc18806b2b0 msgr2=0x7fc1880d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 shutdown_connections 2023-07-06T21:39:02.343 INFO:teuthology.orchestra.run.smithi107.stderr:2023-07-06T21:39:02.340+0000 7fc18d5ff700 1 -- 172.21.15.107:0/1613215817 wait complete. 2023-07-06T21:39:02.359 INFO:tasks.ceph.osd.0:Restarting daemon 2023-07-06T21:39:02.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-07-06T21:39:02.364 INFO:tasks.ceph.osd.0:Started 2023-07-06T21:39:02.364 INFO:tasks.ceph.osd.1:Restarting daemon 2023-07-06T21:39:02.364 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-07-06T21:39:02.368 INFO:tasks.ceph.osd.1:Started 2023-07-06T21:39:02.368 INFO:tasks.ceph.osd.2:Restarting daemon 2023-07-06T21:39:02.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-07-06T21:39:02.372 INFO:tasks.ceph.osd.2:Started 2023-07-06T21:39:02.373 INFO:tasks.ceph.osd.3:Restarting daemon 2023-07-06T21:39:02.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-07-06T21:39:02.378 INFO:tasks.ceph.osd.3:Started 2023-07-06T21:39:02.378 INFO:tasks.ceph.osd.4:Restarting daemon 2023-07-06T21:39:02.378 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-07-06T21:39:02.382 INFO:tasks.ceph.osd.4:Started 2023-07-06T21:39:02.383 INFO:tasks.ceph.osd.5:Restarting daemon 2023-07-06T21:39:02.383 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-07-06T21:39:02.389 INFO:tasks.ceph.osd.5:Started 2023-07-06T21:39:02.389 INFO:tasks.ceph.osd.6:Restarting daemon 2023-07-06T21:39:02.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-07-06T21:39:02.394 INFO:tasks.ceph.osd.6:Started 2023-07-06T21:39:02.394 INFO:tasks.ceph.osd.7:Restarting daemon 2023-07-06T21:39:02.394 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-07-06T21:39:02.399 INFO:tasks.ceph.osd.7:Started 2023-07-06T21:39:02.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.663+0000 7f228b5ff700 1 Processor -- start 2023-07-06T21:39:02.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.664+0000 7f228b5ff700 1 -- start start 2023-07-06T21:39:02.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840d3710 0x7f22840cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228b5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f22840cfa90 0x7f22840d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228b5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840cd910 con 0x7f22840b83c0 2023-07-06T21:39:02.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228b5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22840cfee0 con 0x7f22840cfa90 2023-07-06T21:39:02.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22840d0a80 con 0x7f22840d3710 2023-07-06T21:39:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f2289b9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f22840cfa90 0x7f22840d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228939b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f22840b83c0 0x7f22840cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:38838/0 (socket says 172.21.15.84:38838) 2023-07-06T21:39:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f2288b9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840d3710 0x7f22840cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.665+0000 7f228939b700 1 -- 172.21.15.84:0/3295074824 learned_addr learned my addr 172.21.15.84:0/3295074824 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f2288b9a700 1 -- 172.21.15.84:0/3295074824 >> v2:172.21.15.107:3300/0 conn(0x7f22840cfa90 msgr2=0x7f22840d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f2288b9a700 1 --2- 172.21.15.84:0/3295074824 >> v2:172.21.15.107:3300/0 conn(0x7f22840cfa90 0x7f22840d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f2288b9a700 1 -- 172.21.15.84:0/3295074824 >> v1:172.21.15.107:6789/0 conn(0x7f22840b83c0 legacy=0x7f22840cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f2288b9a700 1 -- 172.21.15.84:0/3295074824 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840b8790 con 0x7f22840d3710 2023-07-06T21:39:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f2289b9c700 1 --2- 172.21.15.84:0/3295074824 >> v2:172.21.15.107:3300/0 conn(0x7f22840cfa90 0x7f22840d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f2288b9a700 1 --2- 172.21.15.84:0/3295074824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840d3710 0x7f22840cf550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2274009840 tx=0x7f2274009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4ed67d90905064 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f227bfff700 1 -- 172.21.15.84:0/3295074824 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274011830 con 0x7f22840d3710 2023-07-06T21:39:02.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f227bfff700 1 -- 172.21.15.84:0/3295074824 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274011990 con 0x7f22840d3710 2023-07-06T21:39:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.666+0000 7f227bfff700 1 -- 172.21.15.84:0/3295074824 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274011d20 con 0x7f22840d3710 2023-07-06T21:39:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 -- 172.21.15.84:0/3295074824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840d3710 msgr2=0x7f22840cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 --2- 172.21.15.84:0/3295074824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840d3710 0x7f22840cf550 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2274009840 tx=0x7f2274009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:02.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 -- 172.21.15.84:0/3295074824 shutdown_connections 2023-07-06T21:39:02.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 --2- 172.21.15.84:0/3295074824 >> v2:172.21.15.107:3300/0 conn(0x7f22840cfa90 0x7f22840d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 --2- 172.21.15.84:0/3295074824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840d3710 0x7f22840cf550 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 -- 172.21.15.84:0/3295074824 >> 172.21.15.84:0/3295074824 conn(0x7f228406b350 msgr2=0x7f228406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:02.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 -- 172.21.15.84:0/3295074824 shutdown_connections 2023-07-06T21:39:02.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.667+0000 7f228b5ff700 1 -- 172.21.15.84:0/3295074824 wait complete. 2023-07-06T21:39:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.669+0000 7f228b5ff700 1 Processor -- start 2023-07-06T21:39:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.669+0000 7f228b5ff700 1 -- start start 2023-07-06T21:39:02.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840b83c0 0x7f22840ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228b5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22840cfa90 0x7f2284159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228b5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 0x7f22841599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228b5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2284155f00 con 0x7f22840d3710 2023-07-06T21:39:02.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228b5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841561e0 con 0x7f22840cfa90 2023-07-06T21:39:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22841564c0 con 0x7f22840b83c0 2023-07-06T21:39:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 0x7f22841599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228939b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840b83c0 0x7f22840ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 0x7f22841599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44556/0 (socket says 172.21.15.84:44556) 2023-07-06T21:39:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f228939b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840b83c0 0x7f22840ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:34068/0 (socket says 172.21.15.84:34068) 2023-07-06T21:39:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 -- 172.21.15.84:0/1575402304 learned_addr learned my addr 172.21.15.84:0/1575402304 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:02.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2288b9a700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22840cfa90 0x7f2284159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 -- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840b83c0 msgr2=0x7f22840ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840b83c0 0x7f22840ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 -- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22840cfa90 msgr2=0x7f2284159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22840cfa90 0x7f2284159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.670+0000 7f2289b9c700 1 -- 172.21.15.84:0/1575402304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c009a70 con 0x7f22840d3710 2023-07-06T21:39:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f2289b9c700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 0x7f22841599c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f227c00efc0 tx=0x7f227c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ca90 con 0x7f22840d3710 2023-07-06T21:39:02.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c00f040 con 0x7f22840d3710 2023-07-06T21:39:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22740094a0 con 0x7f22840d3710 2023-07-06T21:39:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ce30 con 0x7f22840d3710 2023-07-06T21:39:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2284156a80 con 0x7f22840d3710 2023-07-06T21:39:02.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f227c003d20 con 0x7f22840d3710 2023-07-06T21:39:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.671+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f227c014480 con 0x7f22840d3710 2023-07-06T21:39:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.672+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2284156c70 con 0x7f22840d3710 2023-07-06T21:39:02.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.674+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f227c018030 con 0x7f22840d3710 2023-07-06T21:39:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.747+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22841574d0 con 0x7f22840d3710 2023-07-06T21:39:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.748+0000 7f2279ffb700 1 -- 172.21.15.84:0/1575402304 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f227c003fd0 con 0x7f22840d3710 2023-07-06T21:39:02.751 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:02.751 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":10,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:02.333069+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 msgr2=0x7f22841599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 0x7f22841599c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f227c00efc0 tx=0x7f227c00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 shutdown_connections 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22840d3710 0x7f22841599c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22840b83c0 0x7f22840ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 --2- 172.21.15.84:0/1575402304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22840cfa90 0x7f2284159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:02.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 >> 172.21.15.84:0/1575402304 conn(0x7f228406b350 msgr2=0x7f22840b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 shutdown_connections 2023-07-06T21:39:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:02.749+0000 7f228b5ff700 1 -- 172.21.15.84:0/1575402304 wait complete. 2023-07-06T21:39:02.770 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-07-06T21:39:02.770 INFO:tasks.ceph:Waiting for OSDs to come up 2023-07-06T21:39:02.824 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-07-06T21:39:02.822+0000 7f55b05e8640 -1 Falling back to public interface 2023-07-06T21:39:02.865 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:02.863+0000 7f2aaff32200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T21:39:02.900 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:02.897+0000 7f1cd3851200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-06T21:39:02.965 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:02.963+0000 7f2aaff32200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T21:39:02.999 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:02.997+0000 7f1cd3851200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-06T21:39:03.065 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-06T21:39:03.062+0000 7fe02dca4640 -1 Falling back to public interface 2023-07-06T21:39:03.074 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-06T21:39:03.072+0000 7f44a721b640 -1 Falling back to public interface 2023-07-06T21:39:03.083 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-07-06T21:39:03.081+0000 7fd4fcaed640 -1 Falling back to public interface 2023-07-06T21:39:03.095 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-07-06T21:39:03.093+0000 7f7eb0298640 -1 Falling back to public interface 2023-07-06T21:39:03.095 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-06T21:39:03.093+0000 7f66f3ae1640 -1 Falling back to public interface 2023-07-06T21:39:03.100 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-07-06T21:39:03.099+0000 7f4cc7e96640 -1 Falling back to public interface 2023-07-06T21:39:03.122 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-06T21:39:03.120+0000 7f56eb6f4640 -1 Falling back to public interface 2023-07-06T21:39:03.145 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:03.143+0000 7f2aaff32200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T21:39:03.180 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:03.177+0000 7f1cd3851200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-06T21:39:03.444 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:03.442+0000 7f2aaff32200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T21:39:03.475 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:03.473+0000 7f1cd3851200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-06T21:39:03.538 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:03.536+0000 7f2aaff32200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T21:39:03.570 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:03.567+0000 7f1cd3851200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-06T21:39:03.574 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:03.751 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.749+0000 7fee44bff700 1 Processor -- start 2023-07-06T21:39:03.752 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.750+0000 7fee44bff700 1 -- start start 2023-07-06T21:39:03.753 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee44bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee400cffe0 0x7fee40070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:03.753 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee44bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee4006ba10 0x7fee4006be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:03.753 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee44bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee400d05e0 con 0x7fee4015c220 2023-07-06T21:39:03.753 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee44bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400ceae0 con 0x7fee4006ba10 2023-07-06T21:39:03.754 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee44bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee400cb610 con 0x7fee400cffe0 2023-07-06T21:39:03.754 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee4006ba10 0x7fee4006be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:03.754 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fee4006ba10 0x7fee4006be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59506/0 (socket says 172.21.15.84:59506) 2023-07-06T21:39:03.754 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 -- 172.21.15.84:0/1158090910 learned_addr learned my addr 172.21.15.84:0/1158090910 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:03.754 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3dd9b700 1 --2- 172.21.15.84:0/1158090910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee400cffe0 0x7fee40070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:03.755 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 -- 172.21.15.84:0/1158090910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee400cffe0 msgr2=0x7fee40070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:03.755 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 --2- 172.21.15.84:0/1158090910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee400cffe0 0x7fee40070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 -- 172.21.15.84:0/1158090910 >> v1:172.21.15.107:6789/0 conn(0x7fee4015c220 legacy=0x7fee400caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.751+0000 7fee3ed9d700 1 -- 172.21.15.84:0/1158090910 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4006cfd0 con 0x7fee4006ba10 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.752+0000 7fee3ed9d700 1 --2- 172.21.15.84:0/1158090910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4006ba10 0x7fee4006be70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fee38009e50 tx=0x7fee3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cfaeb0546e919c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.752+0000 7fee3d59a700 1 -- 172.21.15.84:0/1158090910 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800e070 con 0x7fee4006ba10 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.752+0000 7fee3d59a700 1 -- 172.21.15.84:0/1158090910 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3800b910 con 0x7fee4006ba10 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.752+0000 7fee3d59a700 1 -- 172.21.15.84:0/1158090910 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3800bcc0 con 0x7fee4006ba10 2023-07-06T21:39:03.756 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.752+0000 7fee44bff700 1 -- 172.21.15.84:0/1158090910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4006ba10 msgr2=0x7fee4006be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.752+0000 7fee44bff700 1 --2- 172.21.15.84:0/1158090910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4006ba10 0x7fee4006be70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fee38009e50 tx=0x7fee3800b330 comp rx=0 tx=0).stop 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.753+0000 7fee44bff700 1 -- 172.21.15.84:0/1158090910 shutdown_connections 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.753+0000 7fee44bff700 1 --2- 172.21.15.84:0/1158090910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee4006ba10 0x7fee4006be70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.753+0000 7fee44bff700 1 --2- 172.21.15.84:0/1158090910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee400cffe0 0x7fee40070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.753+0000 7fee44bff700 1 -- 172.21.15.84:0/1158090910 >> 172.21.15.84:0/1158090910 conn(0x7fee4006e5b0 msgr2=0x7fee4006e980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.753+0000 7fee44bff700 1 -- 172.21.15.84:0/1158090910 shutdown_connections 2023-07-06T21:39:03.757 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.753+0000 7fee44bff700 1 -- 172.21.15.84:0/1158090910 wait complete. 2023-07-06T21:39:03.759 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.754+0000 7fee44bff700 1 Processor -- start 2023-07-06T21:39:03.759 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 -- start start 2023-07-06T21:39:03.760 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4006ba10 0x7fee4015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:03.760 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 0x7fee40146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:03.760 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee4015c220 0x7fee40147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:03.760 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4015f100 con 0x7fee400cffe0 2023-07-06T21:39:03.760 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4015e760 con 0x7fee4006ba10 2023-07-06T21:39:03.760 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.755+0000 7fee44bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4015ea40 con 0x7fee4015c220 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 0x7fee40146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 0x7fee40146d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59514/0 (socket says 172.21.15.84:59514) 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 -- 172.21.15.84:0/277044531 learned_addr learned my addr 172.21.15.84:0/277044531 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3e59c700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4006ba10 0x7fee4015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3ed9d700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee4015c220 0x7fee40147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 -- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee4015c220 msgr2=0x7fee40147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:03.761 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee4015c220 0x7fee40147280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.762 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 -- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4006ba10 msgr2=0x7fee4015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:03.762 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4006ba10 0x7fee4015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.762 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 -- 172.21.15.84:0/277044531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee38009b30 con 0x7fee400cffe0 2023-07-06T21:39:03.762 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.756+0000 7fee3dd9b700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 0x7fee40146d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fee3400e870 tx=0x7fee3400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:03.762 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee34009590 con 0x7fee400cffe0 2023-07-06T21:39:03.762 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34013680 con 0x7fee400cffe0 2023-07-06T21:39:03.763 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee340139f0 con 0x7fee400cffe0 2023-07-06T21:39:03.763 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4015ed20 con 0x7fee400cffe0 2023-07-06T21:39:03.763 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4015cbe0 con 0x7fee400cffe0 2023-07-06T21:39:03.763 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fee34005a20 con 0x7fee400cffe0 2023-07-06T21:39:03.763 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.757+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fee34005cd0 con 0x7fee400cffe0 2023-07-06T21:39:03.763 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.758+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee400741c0 con 0x7fee400cffe0 2023-07-06T21:39:03.764 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.760+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fee34021060 con 0x7fee400cffe0 2023-07-06T21:39:03.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.840+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee400cb8b0 con 0x7fee400cffe0 2023-07-06T21:39:03.843 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.841+0000 7fee2effd700 1 -- 172.21.15.84:0/277044531 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fee34029020 con 0x7fee400cffe0 2023-07-06T21:39:03.843 INFO:teuthology.misc.health.smithi084.stdout: 2023-07-06T21:39:03.843 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":10,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:02.333069+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:03.844 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 msgr2=0x7fee40146d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:03.844 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 0x7fee40146d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fee3400e870 tx=0x7fee3400eb80 comp rx=0 tx=0).stop 2023-07-06T21:39:03.844 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 shutdown_connections 2023-07-06T21:39:03.845 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee4015c220 0x7fee40147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.845 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee4006ba10 0x7fee4015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.845 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 --2- 172.21.15.84:0/277044531 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee400cffe0 0x7fee40146d40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:03.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 >> 172.21.15.84:0/277044531 conn(0x7fee4006e5b0 msgr2=0x7fee400b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:03.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 shutdown_connections 2023-07-06T21:39:03.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:03.842+0000 7fee44bff700 1 -- 172.21.15.84:0/277044531 wait complete. 2023-07-06T21:39:03.858 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T21:39:03.887 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:03.885+0000 7f2aaff32200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T21:39:03.900 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:03.898+0000 7f1cd3851200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-06T21:39:04.166 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:04.164+0000 7f2aaff32200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T21:39:04.177 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:04.175+0000 7f1cd3851200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-06T21:39:04.368 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:04.366+0000 7f2aaff32200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T21:39:04.377 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:04.374+0000 7f1cd3851200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-06T21:39:04.517 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:04.515+0000 7f2aaff32200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T21:39:04.524 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:04.521+0000 7f1cd3851200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-06T21:39:04.704 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:04.702+0000 7f2aaff32200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T21:39:04.708 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:04.706+0000 7f1cd3851200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-06T21:39:05.037 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:05.035+0000 7f2aaff32200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T21:39:05.043 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:05.037+0000 7f1cd3851200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-06T21:39:05.270 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:05.267+0000 7f1cd3851200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T21:39:05.273 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:05.271+0000 7f2aaff32200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-06T21:39:06.104 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:06.102+0000 7f2aaff32200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T21:39:06.132 INFO:tasks.ceph.mgr.x.smithi107.stderr:2023-07-06T21:39:06.129+0000 7f1cd3851200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-06T21:39:06.374 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-07-06T21:39:06.372+0000 7f55b05e8640 -1 osd.0 0 log_to_monitors true 2023-07-06T21:39:06.612 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-07-06T21:39:06.610+0000 7fd4fcaed640 -1 osd.2 0 log_to_monitors true 2023-07-06T21:39:06.629 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-06T21:39:06.626+0000 7f44a721b640 -1 osd.5 0 log_to_monitors true 2023-07-06T21:39:06.653 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-07-06T21:39:06.651+0000 7f7eb0298640 -1 osd.1 0 log_to_monitors true 2023-07-06T21:39:06.661 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-06T21:39:06.659+0000 7fe02dca4640 -1 osd.4 0 log_to_monitors true 2023-07-06T21:39:06.663 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-07-06T21:39:06.661+0000 7f4cc7e96640 -1 osd.3 0 log_to_monitors true 2023-07-06T21:39:06.676 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-06T21:39:06.673+0000 7f66f3ae1640 -1 osd.6 0 log_to_monitors true 2023-07-06T21:39:06.908 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-06T21:39:06.905+0000 7f56eb6f4640 -1 osd.7 0 log_to_monitors true 2023-07-06T21:39:07.126 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:07.124+0000 7f2a639fa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-06T21:39:07.126 INFO:tasks.ceph.mgr.y.smithi084.stderr:2023-07-06T21:39:07.124+0000 7f2a639fa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-07-06T21:39:10.260 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-07-06T21:39:10.258+0000 7fd4eb1f6700 -1 osd.2 0 waiting for initial osdmap 2023-07-06T21:39:10.275 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-06T21:39:10.272+0000 7fe01c3f6700 -1 osd.4 0 waiting for initial osdmap 2023-07-06T21:39:10.280 INFO:tasks.ceph.osd.2.smithi084.stderr:2023-07-06T21:39:10.277+0000 7fd4e31fb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.294 INFO:tasks.ceph.osd.4.smithi107.stderr:2023-07-06T21:39:10.291+0000 7fe0143fb700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.296 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-06T21:39:10.294+0000 7f56d9df6700 -1 osd.7 0 waiting for initial osdmap 2023-07-06T21:39:10.309 INFO:tasks.ceph.osd.7.smithi107.stderr:2023-07-06T21:39:10.307+0000 7f56d1dfb700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.316 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-07-06T21:39:10.314+0000 7f559ebf6700 -1 osd.0 0 waiting for initial osdmap 2023-07-06T21:39:10.321 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-07-06T21:39:10.319+0000 7f4cb65f6700 -1 osd.3 0 waiting for initial osdmap 2023-07-06T21:39:10.354 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-06T21:39:10.334+0000 7f66e21f6700 -1 osd.6 0 waiting for initial osdmap 2023-07-06T21:39:10.354 INFO:tasks.ceph.osd.0.smithi084.stderr:2023-07-06T21:39:10.328+0000 7f5596bfb700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.355 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-06T21:39:10.345+0000 7f44959f6700 -1 osd.5 0 waiting for initial osdmap 2023-07-06T21:39:10.355 INFO:tasks.ceph.osd.3.smithi084.stderr:2023-07-06T21:39:10.336+0000 7f4cae5fb700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.355 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-07-06T21:39:10.337+0000 7f7e9e9f6700 -1 osd.1 0 waiting for initial osdmap 2023-07-06T21:39:10.355 INFO:tasks.ceph.osd.6.smithi107.stderr:2023-07-06T21:39:10.350+0000 7f66da1fb700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.357 INFO:tasks.ceph.osd.1.smithi084.stderr:2023-07-06T21:39:10.355+0000 7f7e969fb700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.362 INFO:tasks.ceph.osd.5.smithi107.stderr:2023-07-06T21:39:10.360+0000 7f448d9fb700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-06T21:39:10.664 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:10.833 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.831+0000 7f2e131ff700 1 Processor -- start 2023-07-06T21:39:10.833 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.831+0000 7f2e131ff700 1 -- start start 2023-07-06T21:39:10.837 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e131ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e0c0d1100 0x7f2e0c0d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:10.837 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e131ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d33c0 0x7f2e0c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:10.837 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e131ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c0d1670 con 0x7f2e0c0d2ab0 2023-07-06T21:39:10.838 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e131ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c0cefa0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.838 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e131ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c0cd210 con 0x7f2e0c0d33c0 2023-07-06T21:39:10.838 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e0bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e0c0d1100 0x7f2e0c0d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:10.839 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e10f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2e0c0d2ab0 0x7f2e0c0b5970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57390/0 (socket says 172.21.15.84:57390) 2023-07-06T21:39:10.839 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e0bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2e0c0d1100 0x7f2e0c0d2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59632/0 (socket says 172.21.15.84:59632) 2023-07-06T21:39:10.839 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.832+0000 7f2e10f9b700 1 -- 172.21.15.84:0/975676556 learned_addr learned my addr 172.21.15.84:0/975676556 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:10.839 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e1179c700 1 --2- 172.21.15.84:0/975676556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d33c0 0x7f2e0c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:10.840 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0bfff700 1 -- 172.21.15.84:0/975676556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d33c0 msgr2=0x7f2e0c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:10.840 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0bfff700 1 --2- 172.21.15.84:0/975676556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d33c0 0x7f2e0c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:10.840 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0bfff700 1 -- 172.21.15.84:0/975676556 >> v1:172.21.15.84:6789/0 conn(0x7f2e0c0d2ab0 legacy=0x7f2e0c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:10.840 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0bfff700 1 -- 172.21.15.84:0/975676556 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c0cd4f0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.841 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0b7fe700 1 -- 172.21.15.84:0/975676556 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750825175 0 0) 0x7f2e0c0d1670 con 0x7f2e0c0d2ab0 2023-07-06T21:39:10.841 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0bfff700 1 --2- 172.21.15.84:0/975676556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c0d2e80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009840 tx=0x7f2df8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e99467729b523 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:10.841 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0b7fe700 1 -- 172.21.15.84:0/975676556 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8011830 con 0x7f2e0c0d1100 2023-07-06T21:39:10.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0b7fe700 1 -- 172.21.15.84:0/975676556 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8011990 con 0x7f2e0c0d1100 2023-07-06T21:39:10.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.833+0000 7f2e0b7fe700 1 -- 172.21.15.84:0/975676556 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8011d20 con 0x7f2e0c0d1100 2023-07-06T21:39:10.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 -- 172.21.15.84:0/975676556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 msgr2=0x7f2e0c0d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:10.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 --2- 172.21.15.84:0/975676556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c0d2e80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009840 tx=0x7f2df8009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:10.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 -- 172.21.15.84:0/975676556 shutdown_connections 2023-07-06T21:39:10.842 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 --2- 172.21.15.84:0/975676556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d33c0 0x7f2e0c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:10.843 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 --2- 172.21.15.84:0/975676556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c0d2e80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:10.843 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 -- 172.21.15.84:0/975676556 >> 172.21.15.84:0/975676556 conn(0x7f2e0c06b3b0 msgr2=0x7f2e0c06b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:10.843 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 -- 172.21.15.84:0/975676556 shutdown_connections 2023-07-06T21:39:10.843 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.834+0000 7f2e131ff700 1 -- 172.21.15.84:0/975676556 wait complete. 2023-07-06T21:39:10.845 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.836+0000 7f2e131ff700 1 Processor -- start 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.837+0000 7f2e131ff700 1 -- start start 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e131ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e131ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d2ab0 0x7f2e0c15da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e131ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 0x7f2e0c15df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e131ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1f2560 con 0x7f2e0c0d1100 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e131ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c15e5a0 con 0x7f2e0c0d33c0 2023-07-06T21:39:10.846 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e131ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c15e880 con 0x7f2e0c0d2ab0 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e10f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e1179c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 0x7f2e0c15df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e0bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d2ab0 0x7f2e0c15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e1179c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 0x7f2e0c15df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:52668/0 (socket says 172.21.15.84:52668) 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.838+0000 7f2e1179c700 1 -- 172.21.15.84:0/3503944256 learned_addr learned my addr 172.21.15.84:0/3503944256 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e10f9b700 1 -- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d2ab0 msgr2=0x7f2e0c15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:10.847 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e10f9b700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d2ab0 0x7f2e0c15da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e10f9b700 1 -- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 msgr2=0x7f2e0c15df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e10f9b700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 0x7f2e0c15df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e10f9b700 1 -- 172.21.15.84:0/3503944256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df80094a0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e1179c700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 0x7f2e0c15df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e10f9b700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c14dc20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c4b0 tx=0x7f2e0000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00004c80 con 0x7f2e0c0d1100 2023-07-06T21:39:10.848 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00004e00 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c1e4be0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00007310 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.839+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1e51a0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.840+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c0741c0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.841+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) v1 ==== 88981+0+0 (secure 0 0 0) 0x7f2e00007470 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.842+0000 7f2e097fa700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2df006ac50 0x7f2df006d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.842+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f2e000880b0 con 0x7f2e0c0d1100 2023-07-06T21:39:10.849 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.843+0000 7f2e0bfff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2df006ac50 0x7f2df006d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:10.850 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.843+0000 7f2e0bfff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2df006ac50 0x7f2df006d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009840 tx=0x7f2df8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:10.850 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:10.847+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2e000541f0 con 0x7f2e0c0d1100 2023-07-06T21:39:11.046 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.044+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e0c0ce950 con 0x7f2e0c0d1100 2023-07-06T21:39:11.047 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.045+0000 7f2e097fa700 1 -- 172.21.15.84:0/3503944256 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2e00057810 con 0x7f2e0c0d1100 2023-07-06T21:39:11.047 INFO:teuthology.misc.health.smithi084.stdout: 2023-07-06T21:39:11.048 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":12,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:08.166468+0000","last_up_change":"0.000000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:11.049 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2df006ac50 msgr2=0x7f2df006d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:11.049 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2df006ac50 0x7f2df006d100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009840 tx=0x7f2df8008030 comp rx=0 tx=0).stop 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 msgr2=0x7f2e0c14dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c14dc20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c4b0 tx=0x7f2e0000c7c0 comp rx=0 tx=0).stop 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 shutdown_connections 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2df006ac50 0x7f2df006d100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0c0d33c0 0x7f2e0c15df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2e0c0d1100 0x7f2e0c14dc20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 --2- 172.21.15.84:0/3503944256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0c0d2ab0 0x7f2e0c15da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:11.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 >> 172.21.15.84:0/3503944256 conn(0x7f2e0c06b3b0 msgr2=0x7f2e0c0b6170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:11.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 shutdown_connections 2023-07-06T21:39:11.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:11.047+0000 7f2e131ff700 1 -- 172.21.15.84:0/3503944256 wait complete. 2023-07-06T21:39:11.064 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-07-06T21:39:17.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:39:17.492 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:39:17.870 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:18.046 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.044+0000 7f8b0c3ff700 1 Processor -- start 2023-07-06T21:39:18.047 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.044+0000 7f8b0c3ff700 1 -- start start 2023-07-06T21:39:18.048 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.045+0000 7f8b0c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d2ab0 0x7f8b040b5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.048 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.045+0000 7f8b0c3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8b040d1100 0x7f8b040d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.048 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.045+0000 7f8b0c3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b040d33c0 0x7f8b040cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.048 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.045+0000 7f8b0c3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040cefa0 con 0x7f8b040d33c0 2023-07-06T21:39:18.048 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.045+0000 7f8b0c3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b040d1670 con 0x7f8b040d1100 2023-07-06T21:39:18.048 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.045+0000 7f8b0c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040cd210 con 0x7f8b040d2ab0 2023-07-06T21:39:18.049 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b040d33c0 0x7f8b040cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.049 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8b040d33c0 0x7f8b040cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45228/0 (socket says 172.21.15.84:45228) 2023-07-06T21:39:18.049 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2927706518 learned_addr learned my addr 172.21.15.84:0/2927706518 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:18.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a19b700 1 --2- 172.21.15.84:0/2927706518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d2ab0 0x7f8b040b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.050 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0999a700 1 --2- 172.21.15.84:0/2927706518 >> v2:172.21.15.107:3300/0 conn(0x7f8b040d1100 0x7f8b040d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2927706518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d2ab0 msgr2=0x7f8b040b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 --2- 172.21.15.84:0/2927706518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d2ab0 0x7f8b040b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2927706518 >> v2:172.21.15.107:3300/0 conn(0x7f8b040d1100 msgr2=0x7f8b040d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 --2- 172.21.15.84:0/2927706518 >> v2:172.21.15.107:3300/0 conn(0x7f8b040d1100 0x7f8b040d2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2927706518 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b040cd4f0 con 0x7f8b040d33c0 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.046+0000 7f8b0a99c700 1 --2- 172.21.15.84:0/2927706518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b040cf3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406c570 tx=0x7f8afc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3129e456a5f4fa3b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:18.051 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b09199700 1 -- 172.21.15.84:0/2927706518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00e070 con 0x7f8b040d33c0 2023-07-06T21:39:18.052 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b09199700 1 -- 172.21.15.84:0/2927706518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc00b9c0 con 0x7f8b040d33c0 2023-07-06T21:39:18.052 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b09199700 1 -- 172.21.15.84:0/2927706518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00bd70 con 0x7f8b040d33c0 2023-07-06T21:39:18.052 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2927706518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 msgr2=0x7f8b040cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.052 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2927706518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b040cf3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406c570 tx=0x7f8afc00b330 comp rx=0 tx=0).stop 2023-07-06T21:39:18.052 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2927706518 shutdown_connections 2023-07-06T21:39:18.052 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2927706518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b040cf3f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.053 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2927706518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d2ab0 0x7f8b040b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.053 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2927706518 >> v2:172.21.15.107:3300/0 conn(0x7f8b040d1100 0x7f8b040d2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.053 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2927706518 >> 172.21.15.84:0/2927706518 conn(0x7f8b0406b3b0 msgr2=0x7f8b0406b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:18.053 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2927706518 shutdown_connections 2023-07-06T21:39:18.053 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.047+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2927706518 wait complete. 2023-07-06T21:39:18.055 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.049+0000 7f8b0c3ff700 1 Processor -- start 2023-07-06T21:39:18.055 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.049+0000 7f8b0c3ff700 1 -- start start 2023-07-06T21:39:18.055 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d1100 0x7f8b0415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.055 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0c3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b040d2ab0 0x7f8b041f1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0c3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b0415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0c3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b040a97b0 con 0x7f8b040d33c0 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0c3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415d1d0 con 0x7f8b040d2ab0 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0415d4b0 con 0x7f8b040d1100 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0a99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b0415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0a99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b0415e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45244/0 (socket says 172.21.15.84:45244) 2023-07-06T21:39:18.056 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2480890005 learned_addr learned my addr 172.21.15.84:0/2480890005 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0a19b700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d1100 0x7f8b0415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.050+0000 7f8b0999a700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b040d2ab0 0x7f8b041f1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d1100 msgr2=0x7f8b0415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0a99c700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d1100 0x7f8b0415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b040d2ab0 msgr2=0x7f8b041f1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0a99c700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b040d2ab0 0x7f8b041f1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.057 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0a99c700 1 -- 172.21.15.84:0/2480890005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc009b30 con 0x7f8b040d33c0 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0999a700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b040d2ab0 0x7f8b041f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0a99c700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b0415e270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406c050 tx=0x7f8afc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc0099b0 con 0x7f8b040d33c0 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8afc004250 con 0x7f8b040d33c0 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0415d790 con 0x7f8b040d33c0 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc004600 con 0x7f8b040d33c0 2023-07-06T21:39:18.058 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.051+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0415dd50 con 0x7f8b040d33c0 2023-07-06T21:39:18.059 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.052+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0415df70 con 0x7f8b040d33c0 2023-07-06T21:39:18.059 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.053+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8afc004760 con 0x7f8b040d33c0 2023-07-06T21:39:18.059 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.054+0000 7f8afaffd700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8ae806ace0 0x7f8ae806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.060 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.054+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8afc08a530 con 0x7f8b040d33c0 2023-07-06T21:39:18.060 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.054+0000 7f8b0a19b700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8ae806ace0 0x7f8ae806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.060 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.055+0000 7f8b0a19b700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8ae806ace0 0x7f8ae806d190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b041f5080 tx=0x7f8b00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:18.061 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.059+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8afc055a80 con 0x7f8b040d33c0 2023-07-06T21:39:18.258 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.256+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b040741c0 con 0x7f8b040d33c0 2023-07-06T21:39:18.259 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.257+0000 7f8afaffd700 1 -- 172.21.15.84:0/2480890005 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f8afc0590a0 con 0x7f8b040d33c0 2023-07-06T21:39:18.259 INFO:teuthology.misc.health.smithi084.stdout: 2023-07-06T21:39:18.260 INFO:teuthology.misc.health.smithi084.stdout:{"epoch":16,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:14.364957+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:18.262 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8ae806ace0 msgr2=0x7f8ae806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.262 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8ae806ace0 0x7f8ae806d190 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8b041f5080 tx=0x7f8b00008040 comp rx=0 tx=0).stop 2023-07-06T21:39:18.262 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 msgr2=0x7f8b0415e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.263 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b0415e270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b0406c050 tx=0x7f8afc004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.263 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 shutdown_connections 2023-07-06T21:39:18.263 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8ae806ace0 0x7f8ae806d190 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.263 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8b040d33c0 0x7f8b0415e270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.263 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8b040d1100 0x7f8b0415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.264 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 --2- 172.21.15.84:0/2480890005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b040d2ab0 0x7f8b041f1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.264 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 >> 172.21.15.84:0/2480890005 conn(0x7f8b0406b3b0 msgr2=0x7f8b040b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:18.264 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.259+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 shutdown_connections 2023-07-06T21:39:18.265 INFO:teuthology.misc.health.smithi084.stderr:2023-07-06T21:39:18.260+0000 7f8b0c3ff700 1 -- 172.21.15.84:0/2480890005 wait complete. 2023-07-06T21:39:18.277 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-07-06T21:39:18.278 INFO:tasks.ceph:Creating RBD pool 2023-07-06T21:39:18.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd pool create rbd 8 2023-07-06T21:39:18.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.462+0000 7fbe727ff700 1 Processor -- start 2023-07-06T21:39:18.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.462+0000 7fbe727ff700 1 -- start start 2023-07-06T21:39:18.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.463+0000 7fbe727ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.463+0000 7fbe727ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbe6c0b1a80 0x7fbe6c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.463+0000 7fbe727ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6c0d1830 con 0x7fbe6c0ceda0 2023-07-06T21:39:18.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.463+0000 7fbe727ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c0cf330 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.463+0000 7fbe727ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0d1270 con 0x7fbe6c0b81c0 2023-07-06T21:39:18.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:36892/0 (socket says 172.21.15.84:36892) 2023-07-06T21:39:18.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 -- 172.21.15.84:0/909135259 learned_addr learned my addr 172.21.15.84:0/909135259 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:18.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6b7fe700 1 --2- 172.21.15.84:0/909135259 >> v2:172.21.15.107:3300/0 conn(0x7fbe6c0b1a80 0x7fbe6c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 -- 172.21.15.84:0/909135259 >> v2:172.21.15.107:3300/0 conn(0x7fbe6c0b1a80 msgr2=0x7fbe6c0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:18.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 --2- 172.21.15.84:0/909135259 >> v2:172.21.15.107:3300/0 conn(0x7fbe6c0b1a80 0x7fbe6c0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 -- 172.21.15.84:0/909135259 >> v1:172.21.15.107:6789/0 conn(0x7fbe6c0ceda0 legacy=0x7fbe6c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.464+0000 7fbe6bfff700 1 -- 172.21.15.84:0/909135259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0d1550 con 0x7fbe6c0b81c0 2023-07-06T21:39:18.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe6bfff700 1 --2- 172.21.15.84:0/909135259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c0cf600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0098f0 tx=0x7fbe5c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4531d409ffe9b677 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:18.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe6affd700 1 -- 172.21.15.84:0/909135259 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c004230 con 0x7fbe6c0b81c0 2023-07-06T21:39:18.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe6affd700 1 -- 172.21.15.84:0/909135259 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c004390 con 0x7fbe6c0b81c0 2023-07-06T21:39:18.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe6affd700 1 -- 172.21.15.84:0/909135259 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0036a0 con 0x7fbe6c0b81c0 2023-07-06T21:39:18.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe727ff700 1 -- 172.21.15.84:0/909135259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 msgr2=0x7fbe6c0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe727ff700 1 --2- 172.21.15.84:0/909135259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c0cf600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0098f0 tx=0x7fbe5c009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:18.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe727ff700 1 -- 172.21.15.84:0/909135259 shutdown_connections 2023-07-06T21:39:18.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe727ff700 1 --2- 172.21.15.84:0/909135259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c0cf600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.465+0000 7fbe727ff700 1 --2- 172.21.15.84:0/909135259 >> v2:172.21.15.107:3300/0 conn(0x7fbe6c0b1a80 0x7fbe6c0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.466+0000 7fbe727ff700 1 -- 172.21.15.84:0/909135259 >> 172.21.15.84:0/909135259 conn(0x7fbe6c06b2b0 msgr2=0x7fbe6c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.466+0000 7fbe727ff700 1 -- 172.21.15.84:0/909135259 shutdown_connections 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.466+0000 7fbe727ff700 1 -- 172.21.15.84:0/909135259 wait complete. 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.467+0000 7fbe727ff700 1 Processor -- start 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.468+0000 7fbe727ff700 1 -- start start 2023-07-06T21:39:18.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.468+0000 7fbe727ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 0x7fbe6c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.468+0000 7fbe727ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.468+0000 7fbe727ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c0ceda0 0x7fbe6c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe727ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0c7e20 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe727ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c1494a0 con 0x7fbe6c0ceda0 2023-07-06T21:39:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe727ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0c7b40 con 0x7fbe6c0b81c0 2023-07-06T21:39:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 0x7fbe6c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 0x7fbe6c0c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45258/0 (socket says 172.21.15.84:45258) 2023-07-06T21:39:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 -- 172.21.15.84:0/3735562324 learned_addr learned my addr 172.21.15.84:0/3735562324 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6b7fe700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe70d9c700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c0ceda0 0x7fbe6c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 -- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 msgr2=0x7fbe6c148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 -- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c0ceda0 msgr2=0x7fbe6c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c0ceda0 0x7fbe6c0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 -- 172.21.15.84:0/3735562324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c009550 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6b7fe700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:18.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.469+0000 7fbe6bfff700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 0x7fbe6c0c9320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009130 tx=0x7fbe5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.470+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c011830 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.470+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c0119b0 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.470+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c011d60 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.470+0000 7fbe727ff700 1 -- 172.21.15.84:0/3735562324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c156bc0 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.470+0000 7fbe727ff700 1 -- 172.21.15.84:0/3735562324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c157180 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.471+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe440052f0 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.472+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbe5c022070 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.472+0000 7fbe68ff9700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe5806ace0 0x7fbe5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:18.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.473+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe5c08aac0 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.473+0000 7fbe6b7fe700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe5806ace0 0x7fbe5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:18.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.473+0000 7fbe6b7fe700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe5806ace0 0x7fbe5806d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c1e6e30 tx=0x7fbe60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:18.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.477+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbe5c056130 con 0x7fbe6c0b1a80 2023-07-06T21:39:18.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:18.674+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fbe44005b20 con 0x7fbe6c0b1a80 2023-07-06T21:39:19.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.504+0000 7fbe68ff9700 1 -- 172.21.15.84:0/3735562324 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fbe5c02a0a0 con 0x7fbe6c0b1a80 2023-07-06T21:39:19.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe5806ace0 msgr2=0x7fbe5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe5806ace0 0x7fbe5806d190 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c1e6e30 tx=0x7fbe60008040 comp rx=0 tx=0).stop 2023-07-06T21:39:19.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 msgr2=0x7fbe6c0c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 0x7fbe6c0c9320 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009130 tx=0x7fbe5c004750 comp rx=0 tx=0).stop 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 shutdown_connections 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe5806ace0 0x7fbe5806d190 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c0ceda0 0x7fbe6c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbe6c0b1a80 0x7fbe6c0c9320 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 --2- 172.21.15.84:0/3735562324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6c0b81c0 0x7fbe6c148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 >> 172.21.15.84:0/3735562324 conn(0x7fbe6c06b2b0 msgr2=0x7fbe6c0b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 shutdown_connections 2023-07-06T21:39:19.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.507+0000 7fbe527fc700 1 -- 172.21.15.84:0/3735562324 wait complete. 2023-07-06T21:39:19.511 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'rbd' created 2023-07-06T21:39:19.531 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-07-06T21:39:19.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.721+0000 7fcfb13ff700 1 Processor -- start 2023-07-06T21:39:19.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.722+0000 7fcfb13ff700 1 -- start start 2023-07-06T21:39:19.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfb13ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfac15fee0 0x7fcfac1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:19.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfb13ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0ceef0 con 0x7fcfac160ac0 2023-07-06T21:39:19.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfb13ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfac0cf450 con 0x7fcfac0d0ea0 2023-07-06T21:39:19.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfb13ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfac160820 con 0x7fcfac15fee0 2023-07-06T21:39:19.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfab7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfac15fee0 0x7fcfac1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:19.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfab7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcfac15fee0 0x7fcfac1602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45274/0 (socket says 172.21.15.84:45274) 2023-07-06T21:39:19.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfab7fe700 1 -- 172.21.15.84:0/285800395 learned_addr learned my addr 172.21.15.84:0/285800395 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:19.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfab7fe700 1 -- 172.21.15.84:0/285800395 >> v1:172.21.15.107:6789/0 conn(0x7fcfac0d0ea0 legacy=0x7fcfac0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfab7fe700 1 -- 172.21.15.84:0/285800395 >> v1:172.21.15.84:6789/0 conn(0x7fcfac160ac0 legacy=0x7fcfac0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfab7fe700 1 -- 172.21.15.84:0/285800395 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac0cf120 con 0x7fcfac15fee0 2023-07-06T21:39:19.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.723+0000 7fcfa9ffb700 1 -- 172.21.15.84:0/285800395 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1054866 0 0) 0x7fcfac0ceef0 con 0x7fcfac160ac0 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.724+0000 7fcfab7fe700 1 --2- 172.21.15.84:0/285800395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac1602e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009e50 tx=0x7fcfa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ef20460ca58778 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.724+0000 7fcfa9ffb700 1 -- 172.21.15.84:0/285800395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400e070 con 0x7fcfac15fee0 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.724+0000 7fcfa9ffb700 1 -- 172.21.15.84:0/285800395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa400b9c0 con 0x7fcfac15fee0 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.724+0000 7fcfa9ffb700 1 -- 172.21.15.84:0/285800395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa400bd70 con 0x7fcfac15fee0 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.724+0000 7fcfb13ff700 1 -- 172.21.15.84:0/285800395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 msgr2=0x7fcfac1602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.724+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/285800395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac1602e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4009e50 tx=0x7fcfa400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:19.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.725+0000 7fcfb13ff700 1 -- 172.21.15.84:0/285800395 shutdown_connections 2023-07-06T21:39:19.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.725+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/285800395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac1602e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.725+0000 7fcfb13ff700 1 -- 172.21.15.84:0/285800395 >> 172.21.15.84:0/285800395 conn(0x7fcfac0715c0 msgr2=0x7fcfac071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:19.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.725+0000 7fcfb13ff700 1 -- 172.21.15.84:0/285800395 shutdown_connections 2023-07-06T21:39:19.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.725+0000 7fcfb13ff700 1 -- 172.21.15.84:0/285800395 wait complete. 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.726+0000 7fcfb13ff700 1 Processor -- start 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.726+0000 7fcfb13ff700 1 -- start start 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.727+0000 7fcfb13ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.727+0000 7fcfb13ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfac160ac0 0x7fcfac0b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.727+0000 7fcfb13ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfac15e960 0x7fcfac1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.727+0000 7fcfb13ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac1f5720 con 0x7fcfac15fee0 2023-07-06T21:39:19.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.727+0000 7fcfb13ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0b8350 con 0x7fcfac160ac0 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.727+0000 7fcfb13ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac0b8630 con 0x7fcfac15e960 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45286/0 (socket says 172.21.15.84:45286) 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 -- 172.21.15.84:0/2128992834 learned_addr learned my addr 172.21.15.84:0/2128992834 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfab7fe700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfac15e960 0x7fcfac1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaa7fc700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfac160ac0 0x7fcfac0b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:19.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 -- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfac15e960 msgr2=0x7fcfac1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfac15e960 0x7fcfac1f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 -- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfac160ac0 msgr2=0x7fcfac0b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:19.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfac160ac0 0x7fcfac0b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:19.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 -- 172.21.15.84:0/2128992834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4009b30 con 0x7fcfac15fee0 2023-07-06T21:39:19.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaa7fc700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfac160ac0 0x7fcfac0b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.728+0000 7fcfaaffd700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac0cbea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c4d0 tx=0x7fcf9c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.729+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0052d0 con 0x7fcfac15fee0 2023-07-06T21:39:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.729+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf9c005430 con 0x7fcfac15fee0 2023-07-06T21:39:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.729+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0038f0 con 0x7fcfac15fee0 2023-07-06T21:39:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.729+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac0b8910 con 0x7fcfac15fee0 2023-07-06T21:39:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.729+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac1f52c0 con 0x7fcfac15fee0 2023-07-06T21:39:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.730+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac1f5860 con 0x7fcfac15fee0 2023-07-06T21:39:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.731+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcf9c003a50 con 0x7fcfac15fee0 2023-07-06T21:39:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.732+0000 7fcf937fe700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcf9806ace0 0x7fcf9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:19.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.732+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5199+0+0 (secure 0 0 0) 0x7fcf9c088600 con 0x7fcfac15fee0 2023-07-06T21:39:19.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.732+0000 7fcfaa7fc700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcf9806ace0 0x7fcf9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:19.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.733+0000 7fcfaa7fc700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcf9806ace0 0x7fcf9806d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ccee0 tx=0x7fcfa0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:19.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.737+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcfac1f5860 con 0x7fcfac15fee0 2023-07-06T21:39:19.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:19.935+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcfac1f5860 con 0x7fcfac15fee0 2023-07-06T21:39:20.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.509+0000 7fcf937fe700 1 -- 172.21.15.84:0/2128992834 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fcfac1f5860 con 0x7fcfac15fee0 2023-07-06T21:39:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.512+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcf9806ace0 msgr2=0x7fcf9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.512+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcf9806ace0 0x7fcf9806d190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcfac0ccee0 tx=0x7fcfa0009040 comp rx=0 tx=0).stop 2023-07-06T21:39:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.512+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 msgr2=0x7fcfac0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.512+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac0cbea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c4d0 tx=0x7fcf9c00c7e0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 shutdown_connections 2023-07-06T21:39:20.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcf9806ace0 0x7fcf9806d190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfac15e960 0x7fcfac1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfac15fee0 0x7fcfac0cbea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 --2- 172.21.15.84:0/2128992834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfac160ac0 0x7fcfac0b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 >> 172.21.15.84:0/2128992834 conn(0x7fcfac0715c0 msgr2=0x7fcfac073480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 shutdown_connections 2023-07-06T21:39:20.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.513+0000 7fcfb13ff700 1 -- 172.21.15.84:0/2128992834 wait complete. 2023-07-06T21:39:20.517 INFO:teuthology.orchestra.run.smithi084.stderr:enabled application 'rbd' on pool 'rbd' 2023-07-06T21:39:20.534 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-07-06T21:39:20.535 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T21:39:20.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T21:39:20.539 INFO:tasks.ceph.mds.a:Started 2023-07-06T21:39:20.539 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T21:39:20.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T21:39:20.543 INFO:tasks.ceph.mds.c:Started 2023-07-06T21:39:20.544 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T21:39:20.544 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T21:39:20.549 INFO:tasks.ceph.mds.b:Started 2023-07-06T21:39:20.549 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T21:39:20.549 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T21:39:20.553 INFO:tasks.ceph.mds.d:Started 2023-07-06T21:39:20.553 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-07-06T21:39:20.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:20.687 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-07-06T21:39:20.690 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-07-06T21:39:20.698 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2023-07-06T21:39:20.701 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-06T21:39:20.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.759+0000 7ff93d9ff700 1 Processor -- start 2023-07-06T21:39:20.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.759+0000 7ff93d9ff700 1 -- start start 2023-07-06T21:39:20.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff93d9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9380b83c0 0x7ff9380cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:20.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff93d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380cfc60 0x7ff9380d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff93d9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380d0a80 con 0x7ff9380d3710 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff93d9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380b8790 con 0x7ff9380b83c0 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff93d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380b4e70 con 0x7ff9380cfc60 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff936ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9380b83c0 0x7ff9380cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff936ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9380b83c0 0x7ff9380cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45310/0 (socket says 172.21.15.84:45310) 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff936ffd700 1 -- 172.21.15.84:0/2580090841 learned_addr learned my addr 172.21.15.84:0/2580090841 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:20.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.760+0000 7ff9377fe700 1 --2- 172.21.15.84:0/2580090841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380cfc60 0x7ff9380d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff936ffd700 1 -- 172.21.15.84:0/2580090841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380cfc60 msgr2=0x7ff9380d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff936ffd700 1 --2- 172.21.15.84:0/2580090841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380cfc60 0x7ff9380d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff936ffd700 1 -- 172.21.15.84:0/2580090841 >> v1:172.21.15.107:6789/0 conn(0x7ff9380d3710 legacy=0x7ff9380cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff935ffb700 1 -- 172.21.15.84:0/2580090841 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664133791 0 0) 0x7ff9380d0a80 con 0x7ff9380d3710 2023-07-06T21:39:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff936ffd700 1 -- 172.21.15.84:0/2580090841 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380b5150 con 0x7ff9380b83c0 2023-07-06T21:39:20.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff936ffd700 1 --2- 172.21.15.84:0/2580090841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380b83c0 0x7ff9380cd200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098f0 tx=0x7ff928009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a74601452d6a3c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff935ffb700 1 -- 172.21.15.84:0/2580090841 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011830 con 0x7ff9380b83c0 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff935ffb700 1 -- 172.21.15.84:0/2580090841 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff928011990 con 0x7ff9380b83c0 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.761+0000 7ff935ffb700 1 -- 172.21.15.84:0/2580090841 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011d20 con 0x7ff9380b83c0 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 -- 172.21.15.84:0/2580090841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380b83c0 msgr2=0x7ff9380cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/2580090841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380b83c0 0x7ff9380cd200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098f0 tx=0x7ff928009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 -- 172.21.15.84:0/2580090841 shutdown_connections 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/2580090841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380cfc60 0x7ff9380d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/2580090841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380b83c0 0x7ff9380cd200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 -- 172.21.15.84:0/2580090841 >> 172.21.15.84:0/2580090841 conn(0x7ff93806b350 msgr2=0x7ff93806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 -- 172.21.15.84:0/2580090841 shutdown_connections 2023-07-06T21:39:20.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.762+0000 7ff93d9ff700 1 -- 172.21.15.84:0/2580090841 wait complete. 2023-07-06T21:39:20.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.764+0000 7ff93d9ff700 1 Processor -- start 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.764+0000 7ff93d9ff700 1 -- start start 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff93d9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380b83c0 0x7ff9381f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff93d9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9380cfc60 0x7ff9380b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff93d9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 0x7ff9381f1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff93d9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93815ea60 con 0x7ff9380d3710 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff93d9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9381f6240 con 0x7ff9380cfc60 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff93d9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93815c5f0 con 0x7ff9380b83c0 2023-07-06T21:39:20.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 0x7ff9381f1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 0x7ff9381f1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45322/0 (socket says 172.21.15.84:45322) 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 -- 172.21.15.84:0/1969289733 learned_addr learned my addr 172.21.15.84:0/1969289733 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff936ffd700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380b83c0 0x7ff9381f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9367fc700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9380cfc60 0x7ff9380b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 -- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380b83c0 msgr2=0x7ff9381f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380b83c0 0x7ff9381f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 -- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9380cfc60 msgr2=0x7ff9380b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9380cfc60 0x7ff9380b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.765+0000 7ff9377fe700 1 -- 172.21.15.84:0/1969289733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff928009550 con 0x7ff9380d3710 2023-07-06T21:39:20.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.766+0000 7ff9377fe700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 0x7ff9381f1200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff93000efc0 tx=0x7ff930009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.766+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930009b20 con 0x7ff9380d3710 2023-07-06T21:39:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.766+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000f040 con 0x7ff9380d3710 2023-07-06T21:39:20.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.766+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93815c8d0 con 0x7ff9380d3710 2023-07-06T21:39:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.766+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000cc90 con 0x7ff9380d3710 2023-07-06T21:39:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.766+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93815ce90 con 0x7ff9380d3710 2023-07-06T21:39:20.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.767+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9380cf650 con 0x7ff9380d3710 2023-07-06T21:39:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.768+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff930014480 con 0x7ff9380d3710 2023-07-06T21:39:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.769+0000 7ff91ffff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff92406ace0 0x7ff92406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.769+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7ff930089b60 con 0x7ff9380d3710 2023-07-06T21:39:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.769+0000 7ff936ffd700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff92406ace0 0x7ff92406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:20.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.770+0000 7ff936ffd700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff92406ace0 0x7ff92406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff928014040 tx=0x7ff928008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:20.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.773+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff930054f50 con 0x7ff9380d3710 2023-07-06T21:39:20.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.976+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9380cf050 con 0x7ff9380d3710 2023-07-06T21:39:20.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.977+0000 7ff91ffff700 1 -- 172.21.15.84:0/1969289733 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+14649 (secure 0 0 0) 0x7ff930018090 con 0x7ff9380d3710 2023-07-06T21:39:20.979 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:20.980 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:20.503127+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff92406ace0 msgr2=0x7ff92406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff92406ace0 0x7ff92406d190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff928014040 tx=0x7ff928008030 comp rx=0 tx=0).stop 2023-07-06T21:39:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 msgr2=0x7ff9381f1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:20.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 0x7ff9381f1200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff93000efc0 tx=0x7ff930009f60 comp rx=0 tx=0).stop 2023-07-06T21:39:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 shutdown_connections 2023-07-06T21:39:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff92406ace0 0x7ff92406d190 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9380d3710 0x7ff9381f1200 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9380b83c0 0x7ff9381f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 --2- 172.21.15.84:0/1969289733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9380cfc60 0x7ff9380b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 >> 172.21.15.84:0/1969289733 conn(0x7ff93806b350 msgr2=0x7ff9380d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 shutdown_connections 2023-07-06T21:39:20.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:20.980+0000 7ff93d9ff700 1 -- 172.21.15.84:0/1969289733 wait complete. 2023-07-06T21:39:21.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:39:21.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:39:21.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.196+0000 7fcec29ff700 1 Processor -- start 2023-07-06T21:39:21.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.196+0000 7fcec29ff700 1 -- start start 2023-07-06T21:39:21.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcec29ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcebc0b81c0 0x7fcebc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcec29ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc0d1830 con 0x7fcebc0b1a80 2023-07-06T21:39:21.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcec29ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc160d40 con 0x7fcebc0ceda0 2023-07-06T21:39:21.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcec29ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc0cf330 con 0x7fcebc0b81c0 2023-07-06T21:39:21.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcebb7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcebc0b1a80 0x7fcebc15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32782/0 (socket says 172.21.15.84:32782) 2023-07-06T21:39:21.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcebbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcebc0b81c0 0x7fcebc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.197+0000 7fcebb7fe700 1 -- 172.21.15.84:0/33017806 learned_addr learned my addr 172.21.15.84:0/33017806 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:21.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebaffd700 1 -- 172.21.15.84:0/33017806 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169399692 0 0) 0x7fcebc0d1830 con 0x7fcebc0b1a80 2023-07-06T21:39:21.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebaffd700 1 -- 172.21.15.84:0/33017806 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea40035b0 con 0x7fcebc0b1a80 2023-07-06T21:39:21.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebbfff700 1 -- 172.21.15.84:0/33017806 >> v1:172.21.15.107:6789/0 conn(0x7fcebc0ceda0 legacy=0x7fcebc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebbfff700 1 -- 172.21.15.84:0/33017806 >> v1:172.21.15.84:6789/0 conn(0x7fcebc0b1a80 legacy=0x7fcebc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebbfff700 1 -- 172.21.15.84:0/33017806 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc0d1270 con 0x7fcebc0b81c0 2023-07-06T21:39:21.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebbfff700 1 --2- 172.21.15.84:0/33017806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc0cf600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fceac009840 tx=0x7fceac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b303a2c89b5c6c45 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:21.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebaffd700 1 -- 172.21.15.84:0/33017806 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac011830 con 0x7fcebc0b81c0 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebaffd700 1 -- 172.21.15.84:0/33017806 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac011990 con 0x7fcebc0b81c0 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.198+0000 7fcebaffd700 1 -- 172.21.15.84:0/33017806 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac011d20 con 0x7fcebc0b81c0 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 -- 172.21.15.84:0/33017806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 msgr2=0x7fcebc0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 --2- 172.21.15.84:0/33017806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc0cf600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fceac009840 tx=0x7fceac009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 -- 172.21.15.84:0/33017806 shutdown_connections 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 --2- 172.21.15.84:0/33017806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc0cf600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 -- 172.21.15.84:0/33017806 >> 172.21.15.84:0/33017806 conn(0x7fcebc06b2b0 msgr2=0x7fcebc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:21.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 -- 172.21.15.84:0/33017806 shutdown_connections 2023-07-06T21:39:21.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.199+0000 7fcec29ff700 1 -- 172.21.15.84:0/33017806 wait complete. 2023-07-06T21:39:21.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.201+0000 7fcec29ff700 1 Processor -- start 2023-07-06T21:39:21.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.201+0000 7fcec29ff700 1 -- start start 2023-07-06T21:39:21.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec29ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcebc0b1a80 0x7fcebc158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec29ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec29ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc0ceda0 0x7fcebc1e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec29ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc1e6900 con 0x7fcebc0b81c0 2023-07-06T21:39:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec29ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc1e6b50 con 0x7fcebc0ceda0 2023-07-06T21:39:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec29ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc1e6e30 con 0x7fcebc0b1a80 2023-07-06T21:39:21.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45350/0 (socket says 172.21.15.84:45350) 2023-07-06T21:39:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 -- 172.21.15.84:0/253261155 learned_addr learned my addr 172.21.15.84:0/253261155 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:21.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcec0f9c700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc0ceda0 0x7fcebc1e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebbfff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcebc0b1a80 0x7fcebc158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 -- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcebc0b1a80 msgr2=0x7fcebc158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcebc0b1a80 0x7fcebc158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 -- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc0ceda0 msgr2=0x7fcebc1e5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc0ceda0 0x7fcebc1e5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.202+0000 7fcebb7fe700 1 -- 172.21.15.84:0/253261155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0094a0 con 0x7fcebc0b81c0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.203+0000 7fcebb7fe700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc160390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fceb000c700 tx=0x7fceb000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.203+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0005d70 con 0x7fcebc0b81c0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.203+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb00070d0 con 0x7fcebc0b81c0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.203+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc1e7110 con 0x7fcebc0b81c0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.203+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb0007460 con 0x7fcebc0b81c0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.203+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc0cb0c0 con 0x7fcebc0b81c0 2023-07-06T21:39:21.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.204+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcebc04e1d0 con 0x7fcebc0b81c0 2023-07-06T21:39:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.205+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fceb00075c0 con 0x7fcebc0b81c0 2023-07-06T21:39:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.206+0000 7fceb8ff9700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcea806ace0 0x7fcea806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.206+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fceb00887a0 con 0x7fcebc0b81c0 2023-07-06T21:39:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.206+0000 7fcebbfff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcea806ace0 0x7fcea806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.207+0000 7fcebbfff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcea806ace0 0x7fcea806d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fceac0090b0 tx=0x7fceac008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:21.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.210+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fceb0053b10 con 0x7fcebc0b81c0 2023-07-06T21:39:21.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.407+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcebc15a690 con 0x7fcebc0b81c0 2023-07-06T21:39:21.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.408+0000 7fceb8ff9700 1 -- 172.21.15.84:0/253261155 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7fceb0057130 con 0x7fcebc0b81c0 2023-07-06T21:39:21.410 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:39:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcea806ace0 msgr2=0x7fcea806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcea806ace0 0x7fcea806d190 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fceac0090b0 tx=0x7fceac008030 comp rx=0 tx=0).stop 2023-07-06T21:39:21.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 msgr2=0x7fcebc160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc160390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fceb000c700 tx=0x7fceb000cac0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 shutdown_connections 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcea806ace0 0x7fcea806d190 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc0ceda0 0x7fcebc1e5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcebc0b1a80 0x7fcebc158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 --2- 172.21.15.84:0/253261155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcebc0b81c0 0x7fcebc160390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 >> 172.21.15.84:0/253261155 conn(0x7fcebc06b2b0 msgr2=0x7fcebc0d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:21.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 shutdown_connections 2023-07-06T21:39:21.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.410+0000 7fcec29ff700 1 -- 172.21.15.84:0/253261155 wait complete. 2023-07-06T21:39:21.430 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:39:21.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.627+0000 7fa5c6dff700 1 Processor -- start 2023-07-06T21:39:21.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.627+0000 7fa5c6dff700 1 -- start start 2023-07-06T21:39:21.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c6dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5c00b81c0 0x7fa5c00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c6dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00b1a80 0x7fa5c00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c6dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c00d1830 con 0x7fa5c00ceda0 2023-07-06T21:39:21.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c6dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c00cf330 con 0x7fa5c00b81c0 2023-07-06T21:39:21.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c6dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c00d1270 con 0x7fa5c00b1a80 2023-07-06T21:39:21.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c4b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5c00b81c0 0x7fa5c00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c4b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa5c00b81c0 0x7fa5c00cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45356/0 (socket says 172.21.15.84:45356) 2023-07-06T21:39:21.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.628+0000 7fa5c4b9b700 1 -- 172.21.15.84:0/1808282967 learned_addr learned my addr 172.21.15.84:0/1808282967 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:21.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5bffff700 1 --2- 172.21.15.84:0/1808282967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00b1a80 0x7fa5c00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5c4b9b700 1 -- 172.21.15.84:0/1808282967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00b1a80 msgr2=0x7fa5c00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5c4b9b700 1 --2- 172.21.15.84:0/1808282967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00b1a80 0x7fa5c00cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5c4b9b700 1 -- 172.21.15.84:0/1808282967 >> v1:172.21.15.107:6789/0 conn(0x7fa5c00ceda0 legacy=0x7fa5c015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5c4b9b700 1 -- 172.21.15.84:0/1808282967 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c00d1550 con 0x7fa5c00b81c0 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5c4b9b700 1 --2- 172.21.15.84:0/1808282967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c00cf600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40098f0 tx=0x7fa5b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31f77e261755a7e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5bf7fe700 1 -- 172.21.15.84:0/1808282967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400b8a0 con 0x7fa5c00b81c0 2023-07-06T21:39:21.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5bf7fe700 1 -- 172.21.15.84:0/1808282967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b400ba00 con 0x7fa5c00b81c0 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.629+0000 7fa5bf7fe700 1 -- 172.21.15.84:0/1808282967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400bd90 con 0x7fa5c00b81c0 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 -- 172.21.15.84:0/1808282967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 msgr2=0x7fa5c00cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/1808282967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c00cf600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40098f0 tx=0x7fa5b4009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 -- 172.21.15.84:0/1808282967 shutdown_connections 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/1808282967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c00cf600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/1808282967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00b1a80 0x7fa5c00cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 -- 172.21.15.84:0/1808282967 >> 172.21.15.84:0/1808282967 conn(0x7fa5c006b2b0 msgr2=0x7fa5c006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:21.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 -- 172.21.15.84:0/1808282967 shutdown_connections 2023-07-06T21:39:21.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.630+0000 7fa5c6dff700 1 -- 172.21.15.84:0/1808282967 wait complete. 2023-07-06T21:39:21.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.632+0000 7fa5c6dff700 1 Processor -- start 2023-07-06T21:39:21.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.632+0000 7fa5c6dff700 1 -- start start 2023-07-06T21:39:21.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5c6dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c00b1a80 0x7fa5c014b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5c6dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c01495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5c6dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00ceda0 0x7fa5c0149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5c6dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0149130 con 0x7fa5c00b81c0 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5c6dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0148520 con 0x7fa5c00b1a80 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5c6dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c0148800 con 0x7fa5c00ceda0 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c01495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c01495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45364/0 (socket says 172.21.15.84:45364) 2023-07-06T21:39:21.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 -- 172.21.15.84:0/2885729289 learned_addr learned my addr 172.21.15.84:0/2885729289 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 -- 172.21.15.84:0/2885729289 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00ceda0 msgr2=0x7fa5c0149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00ceda0 0x7fa5c0149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 -- 172.21.15.84:0/2885729289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c00b1a80 msgr2=0x7fa5c014b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c00b1a80 0x7fa5c014b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 -- 172.21.15.84:0/2885729289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4009550 con 0x7fa5c00b81c0 2023-07-06T21:39:21.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.633+0000 7fa5bffff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c01495c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e810 tx=0x7fa5ac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.634+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac013610 con 0x7fa5c00b81c0 2023-07-06T21:39:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.634+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac013770 con 0x7fa5c00b81c0 2023-07-06T21:39:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.634+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac013b20 con 0x7fa5c00b81c0 2023-07-06T21:39:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.634+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c0148ae0 con 0x7fa5c00b81c0 2023-07-06T21:39:21.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.634+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c015b900 con 0x7fa5c00b81c0 2023-07-06T21:39:21.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.635+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c004e1d0 con 0x7fa5c00b81c0 2023-07-06T21:39:21.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.636+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa5ac013c80 con 0x7fa5c00b81c0 2023-07-06T21:39:21.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.637+0000 7fa5bd7fa700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5a406ace0 0x7fa5a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.637+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa5ac088e50 con 0x7fa5c00b81c0 2023-07-06T21:39:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.637+0000 7fa5c4b9b700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5a406ace0 0x7fa5a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.638+0000 7fa5c4b9b700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5a406ace0 0x7fa5a406d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5c01f5550 tx=0x7fa5b4010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:21.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.641+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa5ac0541c0 con 0x7fa5c00b81c0 2023-07-06T21:39:21.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.840+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa5c0149270 con 0x7fa5c00b81c0 2023-07-06T21:39:21.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.841+0000 7fa5bd7fa700 1 -- 172.21.15.84:0/2885729289 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa5c0149270 con 0x7fa5c00b81c0 2023-07-06T21:39:21.843 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5a406ace0 msgr2=0x7fa5a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5a406ace0 0x7fa5a406d190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa5c01f5550 tx=0x7fa5b4010000 comp rx=0 tx=0).stop 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 msgr2=0x7fa5c01495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c01495c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00e810 tx=0x7fa5ac00ebd0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 shutdown_connections 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5a406ace0 0x7fa5a406d190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa5c00ceda0 0x7fa5c0149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5c00b1a80 0x7fa5c014b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 --2- 172.21.15.84:0/2885729289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa5c00b81c0 0x7fa5c01495c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:21.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 >> 172.21.15.84:0/2885729289 conn(0x7fa5c006b2b0 msgr2=0x7fa5c00d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:21.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.843+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 shutdown_connections 2023-07-06T21:39:21.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:21.844+0000 7fa5c6dff700 1 -- 172.21.15.84:0/2885729289 wait complete. 2023-07-06T21:39:21.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:39:22.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.054+0000 7fb2fa7ff700 1 Processor -- start 2023-07-06T21:39:22.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.055+0000 7fb2fa7ff700 1 -- start start 2023-07-06T21:39:22.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2fa7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2f40b83c0 0x7fb2f40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2fa7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2fa7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f40cfee0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2fa7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f40d0a80 con 0x7fb2f40b83c0 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2fa7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f40b8790 con 0x7fb2f40d3710 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2f40b83c0 0x7fb2f40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f8d9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb2f40cfa90 0x7fb2f415feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32798/0 (socket says 172.21.15.84:32798) 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f8d9c700 1 -- 172.21.15.84:0/4269312195 learned_addr learned my addr 172.21.15.84:0/4269312195 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f37fe700 1 --2- 172.21.15.84:0/4269312195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f3fff700 1 -- 172.21.15.84:0/4269312195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 msgr2=0x7fb2f40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f3fff700 1 --2- 172.21.15.84:0/4269312195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f40cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f3fff700 1 -- 172.21.15.84:0/4269312195 >> v1:172.21.15.84:6789/0 conn(0x7fb2f40cfa90 legacy=0x7fb2f415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f3fff700 1 -- 172.21.15.84:0/4269312195 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f40b4e70 con 0x7fb2f40b83c0 2023-07-06T21:39:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.056+0000 7fb2f2ffd700 1 -- 172.21.15.84:0/4269312195 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529103781 0 0) 0x7fb2ec001490 con 0x7fb2f40cfa90 2023-07-06T21:39:22.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2f3fff700 1 --2- 172.21.15.84:0/4269312195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40b83c0 0x7fb2f40cd200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009840 tx=0x7fb2e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08b4ebef1b29d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2f2ffd700 1 -- 172.21.15.84:0/4269312195 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4011830 con 0x7fb2f40b83c0 2023-07-06T21:39:22.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2f2ffd700 1 -- 172.21.15.84:0/4269312195 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4011990 con 0x7fb2f40b83c0 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2f2ffd700 1 -- 172.21.15.84:0/4269312195 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4011d20 con 0x7fb2f40b83c0 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/4269312195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40b83c0 msgr2=0x7fb2f40cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/4269312195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40b83c0 0x7fb2f40cd200 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009840 tx=0x7fb2e4009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/4269312195 shutdown_connections 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/4269312195 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40b83c0 0x7fb2f40cd200 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.057+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/4269312195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f40cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.058+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/4269312195 >> 172.21.15.84:0/4269312195 conn(0x7fb2f406b350 msgr2=0x7fb2f406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.058+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/4269312195 shutdown_connections 2023-07-06T21:39:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.058+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/4269312195 wait complete. 2023-07-06T21:39:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.059+0000 7fb2fa7ff700 1 Processor -- start 2023-07-06T21:39:22.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.059+0000 7fb2fa7ff700 1 -- start start 2023-07-06T21:39:22.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.060+0000 7fb2fa7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f40b83c0 0x7fb2f4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.060+0000 7fb2fa7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 0x7fb2f4154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.060+0000 7fb2fa7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f4155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.060+0000 7fb2fa7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4155f20 con 0x7fb2f40cfa90 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.060+0000 7fb2fa7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4155960 con 0x7fb2f40b83c0 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.060+0000 7fb2fa7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f4155c40 con 0x7fb2f40d3710 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 0x7fb2f4154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 0x7fb2f4154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45374/0 (socket says 172.21.15.84:45374) 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 -- 172.21.15.84:0/2931148269 learned_addr learned my addr 172.21.15.84:0/2931148269 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f8d9c700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f4155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f3fff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f40b83c0 0x7fb2f4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 -- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 msgr2=0x7fb2f4155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f4155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 -- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f40b83c0 msgr2=0x7fb2f4150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f40b83c0 0x7fb2f4150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 -- 172.21.15.84:0/2931148269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40094a0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f8d9c700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f4155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f37fe700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 0x7fb2f4154df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800d800 tx=0x7fb2e800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80125f0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8012750 con 0x7fb2f40cfa90 2023-07-06T21:39:22.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f4156270 con 0x7fb2f40cfa90 2023-07-06T21:39:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.061+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8012ae0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.062+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f4154490 con 0x7fb2f40cfa90 2023-07-06T21:39:22.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.062+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f41560d0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.063+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb2e8012c40 con 0x7fb2f40cfa90 2023-07-06T21:39:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.064+0000 7fb2f0ff9700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb2e006ace0 0x7fb2e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.064+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb2e8089290 con 0x7fb2f40cfa90 2023-07-06T21:39:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.064+0000 7fb2f3fff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb2e006ace0 0x7fb2e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.065+0000 7fb2f3fff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb2e006ace0 0x7fb2e006d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40090b0 tx=0x7fb2e401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.068+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb2f41560d0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.309+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2f41560d0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.310+0000 7fb2f0ff9700 1 -- 172.21.15.84:0/2931148269 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fb2f41560d0 con 0x7fb2f40cfa90 2023-07-06T21:39:22.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:22.312 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-06T21:39:22.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb2e006ace0 msgr2=0x7fb2e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb2e006ace0 0x7fb2e006d190 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40090b0 tx=0x7fb2e401c040 comp rx=0 tx=0).stop 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 msgr2=0x7fb2f4154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 0x7fb2f4154df0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800d800 tx=0x7fb2e800db10 comp rx=0 tx=0).stop 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 shutdown_connections 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb2e006ace0 0x7fb2e006d190 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2f40d3710 0x7fb2f4155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2f40b83c0 0x7fb2f4150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 --2- 172.21.15.84:0/2931148269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2f40cfa90 0x7fb2f4154df0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 >> 172.21.15.84:0/2931148269 conn(0x7fb2f406b350 msgr2=0x7fb2f40cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:22.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 shutdown_connections 2023-07-06T21:39:22.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.312+0000 7fb2fa7ff700 1 -- 172.21.15.84:0/2931148269 wait complete. 2023-07-06T21:39:22.316 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2023-07-06T21:39:22.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:22.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.523+0000 7f70055ff700 1 Processor -- start 2023-07-06T21:39:22.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.524+0000 7f70055ff700 1 -- start start 2023-07-06T21:39:22.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f70055ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f70000d3710 0x7f70000cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f70055ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f70000cfa90 0x7f70000d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f70055ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70000cd910 con 0x7f70000b83c0 2023-07-06T21:39:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f70055ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70000cff10 con 0x7f70000d3710 2023-07-06T21:39:22.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f70055ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70000d0a80 con 0x7f70000cfa90 2023-07-06T21:39:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6ffe7fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f70000d3710 0x7f70000cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6ffeffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f70000b83c0 0x7f70000cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:32812/0 (socket says 172.21.15.84:32812) 2023-07-06T21:39:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6ffeffd700 1 -- 172.21.15.84:0/697670918 learned_addr learned my addr 172.21.15.84:0/697670918 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/697670918 >> v2:172.21.15.107:3300/0 conn(0x7f70000cfa90 0x7f70000d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6ffe7fc700 1 -- 172.21.15.84:0/697670918 >> v2:172.21.15.107:3300/0 conn(0x7f70000cfa90 msgr2=0x7f70000d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6ffe7fc700 1 --2- 172.21.15.84:0/697670918 >> v2:172.21.15.107:3300/0 conn(0x7f70000cfa90 0x7f70000d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.525+0000 7f6ffe7fc700 1 -- 172.21.15.84:0/697670918 >> v1:172.21.15.84:6789/0 conn(0x7f70000b83c0 legacy=0x7f70000cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f6ffe7fc700 1 -- 172.21.15.84:0/697670918 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000b8790 con 0x7f70000d3710 2023-07-06T21:39:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f6ffdffb700 1 -- 172.21.15.84:0/697670918 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415263058 0 0) 0x7f6ff0001110 con 0x7f70000b83c0 2023-07-06T21:39:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f6ffe7fc700 1 --2- 172.21.15.84:0/697670918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000cf550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f70000b1b00 tx=0x7f6ff4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e8506d16526200 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f6ffdffb700 1 -- 172.21.15.84:0/697670918 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4011830 con 0x7f70000d3710 2023-07-06T21:39:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f6ffdffb700 1 -- 172.21.15.84:0/697670918 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4011990 con 0x7f70000d3710 2023-07-06T21:39:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f6ffdffb700 1 -- 172.21.15.84:0/697670918 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4011d20 con 0x7f70000d3710 2023-07-06T21:39:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f70055ff700 1 -- 172.21.15.84:0/697670918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 msgr2=0x7f70000cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.526+0000 7f70055ff700 1 --2- 172.21.15.84:0/697670918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000cf550 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f70000b1b00 tx=0x7f6ff4009b20 comp rx=0 tx=0).stop 2023-07-06T21:39:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.527+0000 7f70055ff700 1 -- 172.21.15.84:0/697670918 shutdown_connections 2023-07-06T21:39:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.527+0000 7f70055ff700 1 --2- 172.21.15.84:0/697670918 >> v2:172.21.15.107:3300/0 conn(0x7f70000cfa90 0x7f70000d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.527+0000 7f70055ff700 1 --2- 172.21.15.84:0/697670918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000cf550 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.527+0000 7f70055ff700 1 -- 172.21.15.84:0/697670918 >> 172.21.15.84:0/697670918 conn(0x7f700006b350 msgr2=0x7f700006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.527+0000 7f70055ff700 1 -- 172.21.15.84:0/697670918 shutdown_connections 2023-07-06T21:39:22.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.527+0000 7f70055ff700 1 -- 172.21.15.84:0/697670918 wait complete. 2023-07-06T21:39:22.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.528+0000 7f70055ff700 1 Processor -- start 2023-07-06T21:39:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.528+0000 7f70055ff700 1 -- start start 2023-07-06T21:39:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.529+0000 7f70055ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70000b83c0 0x7f7000153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.529+0000 7f70055ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70000cfa90 0x7f70000b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.529+0000 7f70055ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.529+0000 7f70055ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70001f3370 con 0x7f70000d3710 2023-07-06T21:39:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.529+0000 7f70055ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700015e610 con 0x7f70000b83c0 2023-07-06T21:39:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.529+0000 7f70055ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70001f3090 con 0x7f70000cfa90 2023-07-06T21:39:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45398/0 (socket says 172.21.15.84:45398) 2023-07-06T21:39:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 -- 172.21.15.84:0/3156448915 learned_addr learned my addr 172.21.15.84:0/3156448915 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6ffeffd700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70000b83c0 0x7f7000153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6ffe7fc700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70000cfa90 0x7f70000b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 -- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70000cfa90 msgr2=0x7f70000b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70000cfa90 0x7f70000b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 -- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70000b83c0 msgr2=0x7f7000153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70000b83c0 0x7f7000153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 -- 172.21.15.84:0/3156448915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40094a0 con 0x7f70000d3710 2023-07-06T21:39:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6ffe7fc700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70000cfa90 0x7f70000b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000b6f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007e60 tx=0x7f6ff800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff800ed80 con 0x7f70000d3710 2023-07-06T21:39:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8014420 con 0x7f70000d3710 2023-07-06T21:39:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70000cb7d0 con 0x7f70000d3710 2023-07-06T21:39:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.530+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80147d0 con 0x7f70000d3710 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.531+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70000cbd90 con 0x7f70000d3710 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.532+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70000cb630 con 0x7f70000d3710 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.532+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ff8014930 con 0x7f70000d3710 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.534+0000 7f6fe77fe700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fec06ace0 0x7f6fec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.534+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6ff8089870 con 0x7f70000d3710 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.534+0000 7f6ffeffd700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fec06ace0 0x7f6fec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.535+0000 7f6ffeffd700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fec06ace0 0x7f6fec06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0000f60 tx=0x7f6ff0007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.538+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f70000cb630 con 0x7f70000d3710 2023-07-06T21:39:22.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.746+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70000cb630 con 0x7f70000d3710 2023-07-06T21:39:22.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.747+0000 7f6fe77fe700 1 -- 172.21.15.84:0/3156448915 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14636 (secure 0 0 0) 0x7f70000cb630 con 0x7f70000d3710 2023-07-06T21:39:22.749 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:22.749 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":19,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:21.508772+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:22.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fec06ace0 msgr2=0x7f6fec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fec06ace0 0x7f6fec06d190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0000f60 tx=0x7f6ff0007c90 comp rx=0 tx=0).stop 2023-07-06T21:39:22.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 msgr2=0x7f70000b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000b6f00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8007e60 tx=0x7f6ff800e3c0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 shutdown_connections 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fec06ace0 0x7f6fec06d190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70000d3710 0x7f70000b6f00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70000b83c0 0x7f7000153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 --2- 172.21.15.84:0/3156448915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70000cfa90 0x7f70000b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 >> 172.21.15.84:0/3156448915 conn(0x7f700006b350 msgr2=0x7f70000d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 shutdown_connections 2023-07-06T21:39:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.749+0000 7f70055ff700 1 -- 172.21.15.84:0/3156448915 wait complete. 2023-07-06T21:39:22.769 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:39:22.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:39:22.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.964+0000 7f207a7ff700 1 Processor -- start 2023-07-06T21:39:22.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.964+0000 7f207a7ff700 1 -- start start 2023-07-06T21:39:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f207a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 0x7f20740cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f207a7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740d1830 con 0x7f20740ceda0 2023-07-06T21:39:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f207a7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2074160d40 con 0x7f20740b1a80 2023-07-06T21:39:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f207a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740cf330 con 0x7f20740b81c0 2023-07-06T21:39:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f2078d9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f20740ceda0 0x7f2074160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52026/0 (socket says 172.21.15.84:52026) 2023-07-06T21:39:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f2078d9c700 1 -- 172.21.15.84:0/1534941143 learned_addr learned my addr 172.21.15.84:0/1534941143 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:22.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f2073fff700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 0x7f20740cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444365806 0 0) 0x7f20740d1830 con 0x7f20740ceda0 2023-07-06T21:39:22.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.966+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f205c0035b0 con 0x7f20740ceda0 2023-07-06T21:39:22.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318076483 0 0) 0x7f2074160d40 con 0x7f20740b1a80 2023-07-06T21:39:22.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20740d1830 con 0x7f20740b1a80 2023-07-06T21:39:22.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4002575520 0 0) 0x7f205c0035b0 con 0x7f20740ceda0 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 msgr2=0x7f20740cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 0x7f20740cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 >> v1:172.21.15.107:6789/0 conn(0x7f20740b1a80 legacy=0x7f207415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740d1270 con 0x7f20740ceda0 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f206c0019f0 con 0x7f20740ceda0 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 >> v1:172.21.15.84:6789/0 conn(0x7f20740ceda0 legacy=0x7f2074160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c003bc0 0x7f205c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c004850 0x7f205c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 0x7f205c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740cf330 con 0x7f205c004850 2023-07-06T21:39:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00bac0 con 0x7f205c003bc0 2023-07-06T21:39:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205c00bc30 con 0x7f20740b81c0 2023-07-06T21:39:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f20737fe700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c004850 0x7f205c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2073fff700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c003bc0 0x7f205c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f2078d9c700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 0x7f205c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.967+0000 7f20737fe700 1 -- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 msgr2=0x7f205c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f20737fe700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b81c0 0x7f205c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f20737fe700 1 -- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c003bc0 msgr2=0x7f205c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f20737fe700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c003bc0 0x7f205c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f20737fe700 1 -- 172.21.15.84:0/1534941143 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c0035b0 con 0x7f205c004850 2023-07-06T21:39:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f20737fe700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c004850 0x7f205c004cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2068001940 tx=0x7f205c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916800024014af9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068001430 con 0x7f205c004850 2023-07-06T21:39:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068001d70 con 0x7f205c004850 2023-07-06T21:39:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f2072ffd700 1 -- 172.21.15.84:0/1534941143 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068005510 con 0x7f205c004850 2023-07-06T21:39:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f207a7ff700 1 -- 172.21.15.84:0/1534941143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c004850 msgr2=0x7f205c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f207a7ff700 1 --2- 172.21.15.84:0/1534941143 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f205c004850 0x7f205c004cc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2068001940 tx=0x7f205c005880 comp rx=0 tx=0).stop 2023-07-06T21:39:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.968+0000 7f2078d9c700 1 -- 172.21.15.84:0/1534941143 reap_dead start 2023-07-06T21:39:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.969+0000 7f207a7ff700 1 -- 172.21.15.84:0/1534941143 shutdown_connections 2023-07-06T21:39:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.969+0000 7f207a7ff700 1 -- 172.21.15.84:0/1534941143 >> 172.21.15.84:0/1534941143 conn(0x7f207406b2b0 msgr2=0x7f207406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:22.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.969+0000 7f207a7ff700 1 -- 172.21.15.84:0/1534941143 shutdown_connections 2023-07-06T21:39:22.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.969+0000 7f207a7ff700 1 -- 172.21.15.84:0/1534941143 wait complete. 2023-07-06T21:39:22.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.970+0000 7f207a7ff700 1 Processor -- start 2023-07-06T21:39:22.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.971+0000 7f207a7ff700 1 -- start start 2023-07-06T21:39:22.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.971+0000 7f207a7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c004850 0x7f20740cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.971+0000 7f207a7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 0x7f20740b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.971+0000 7f207a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b0fe0 0x7f20740c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f207a7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740b1460 con 0x7f20741564e0 2023-07-06T21:39:22.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f207a7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740ccca0 con 0x7f205c004850 2023-07-06T21:39:22.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f207a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20741579d0 con 0x7f20740b0fe0 2023-07-06T21:39:22.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 0x7f20740b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 0x7f20740b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38170/0 (socket says 172.21.15.84:38170) 2023-07-06T21:39:22.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 -- 172.21.15.84:0/3522112863 learned_addr learned my addr 172.21.15.84:0/3522112863 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:22.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f2073fff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c004850 0x7f20740cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f2078d9c700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b0fe0 0x7f20740c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 -- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b0fe0 msgr2=0x7f20740c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b0fe0 0x7f20740c9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 -- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c004850 msgr2=0x7f20740cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:22.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c004850 0x7f20740cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:22.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 -- 172.21.15.84:0/3522112863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207415ecc0 con 0x7f20741564e0 2023-07-06T21:39:22.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.972+0000 7f20737fe700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 0x7f20740b0aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f206800c3c0 tx=0x7f20680012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.973+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068006c70 con 0x7f20741564e0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.973+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068006dd0 con 0x7f20741564e0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.973+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207415efa0 con 0x7f20741564e0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.973+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068007280 con 0x7f20741564e0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.973+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207415f560 con 0x7f20741564e0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.974+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20740b13e0 con 0x7f20741564e0 2023-07-06T21:39:22.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.975+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f20680073e0 con 0x7f20741564e0 2023-07-06T21:39:22.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.975+0000 7f2070ff9700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:22.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.976+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2068089ec0 con 0x7f20741564e0 2023-07-06T21:39:22.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.976+0000 7f2073fff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:22.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.976+0000 7f2073fff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20740b3660 tx=0x7f2064009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:22.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:22.980+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2068055370 con 0x7f20741564e0 2023-07-06T21:39:23.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.174+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f207404e1d0 con 0x7f20741564e0 2023-07-06T21:39:23.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.175+0000 7f2070ff9700 1 -- 172.21.15.84:0/3522112863 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20680170a0 con 0x7f20741564e0 2023-07-06T21:39:23.177 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:39:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 msgr2=0x7f206006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f20740b3660 tx=0x7f2064009210 comp rx=0 tx=0).stop 2023-07-06T21:39:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 msgr2=0x7f20740b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 0x7f20740b0aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f206800c3c0 tx=0x7f20680012d0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 shutdown_connections 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20740b0fe0 0x7f20740c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f205c004850 0x7f20740cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 --2- 172.21.15.84:0/3522112863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20741564e0 0x7f20740b0aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 >> 172.21.15.84:0/3522112863 conn(0x7f207406b2b0 msgr2=0x7f20740715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 shutdown_connections 2023-07-06T21:39:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.177+0000 7f207a7ff700 1 -- 172.21.15.84:0/3522112863 wait complete. 2023-07-06T21:39:23.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:39:23.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.390+0000 7fb4469ff700 1 Processor -- start 2023-07-06T21:39:23.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.390+0000 7fb4469ff700 1 -- start start 2023-07-06T21:39:23.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb4469ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b81c0 0x7fb4400cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb4469ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4400b1a80 0x7fb4400cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb4469ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4400d1830 con 0x7fb4400ceda0 2023-07-06T21:39:23.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb4469ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4400cf330 con 0x7fb4400b1a80 2023-07-06T21:39:23.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb4469ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400d1270 con 0x7fb4400b81c0 2023-07-06T21:39:23.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb437fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4400b1a80 0x7fb4400cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb444f9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb4400ceda0 0x7fb44015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52038/0 (socket says 172.21.15.84:52038) 2023-07-06T21:39:23.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb444f9c700 1 -- 172.21.15.84:0/2616414852 learned_addr learned my addr 172.21.15.84:0/2616414852 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.391+0000 7fb43ffff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b81c0 0x7fb4400cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb43f7fe700 1 -- 172.21.15.84:0/2616414852 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527817984 0 0) 0x7fb4400d1830 con 0x7fb4400ceda0 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb43f7fe700 1 -- 172.21.15.84:0/2616414852 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb42c0035b0 con 0x7fb4400ceda0 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb437fff700 1 -- 172.21.15.84:0/2616414852 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b81c0 msgr2=0x7fb4400cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb437fff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b81c0 0x7fb4400cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb437fff700 1 -- 172.21.15.84:0/2616414852 >> v1:172.21.15.84:6789/0 conn(0x7fb4400ceda0 legacy=0x7fb44015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb437fff700 1 -- 172.21.15.84:0/2616414852 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400d1550 con 0x7fb4400b1a80 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb43ffff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b81c0 0x7fb4400cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb437fff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b1a80 0x7fb4400cfb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb428009840 tx=0x7fb428009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a455cd176c128e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb43f7fe700 1 -- 172.21.15.84:0/2616414852 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428011830 con 0x7fb4400b1a80 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb43f7fe700 1 -- 172.21.15.84:0/2616414852 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428011990 con 0x7fb4400b1a80 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.392+0000 7fb43f7fe700 1 -- 172.21.15.84:0/2616414852 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428011d20 con 0x7fb4400b1a80 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 -- 172.21.15.84:0/2616414852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b1a80 msgr2=0x7fb4400cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b1a80 0x7fb4400cfb40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb428009840 tx=0x7fb428009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 -- 172.21.15.84:0/2616414852 shutdown_connections 2023-07-06T21:39:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b81c0 0x7fb4400cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 --2- 172.21.15.84:0/2616414852 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b1a80 0x7fb4400cfb40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 -- 172.21.15.84:0/2616414852 >> 172.21.15.84:0/2616414852 conn(0x7fb44006b2b0 msgr2=0x7fb44006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 -- 172.21.15.84:0/2616414852 shutdown_connections 2023-07-06T21:39:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.393+0000 7fb4469ff700 1 -- 172.21.15.84:0/2616414852 wait complete. 2023-07-06T21:39:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.395+0000 7fb4469ff700 1 Processor -- start 2023-07-06T21:39:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.396+0000 7fb4469ff700 1 -- start start 2023-07-06T21:39:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb4469ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b1a80 0x7fb440158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb4469ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 0x7fb4400cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb4469ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4400ceda0 0x7fb440156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb4469ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4401577c0 con 0x7fb4400b81c0 2023-07-06T21:39:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb4469ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400cc320 con 0x7fb4400ceda0 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb4469ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb440156400 con 0x7fb4400b1a80 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb437fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 0x7fb4400cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb437fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 0x7fb4400cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38190/0 (socket says 172.21.15.84:38190) 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb437fff700 1 -- 172.21.15.84:0/3570955363 learned_addr learned my addr 172.21.15.84:0/3570955363 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb43ffff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b1a80 0x7fb440158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.397+0000 7fb444f9c700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4400ceda0 0x7fb440156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb437fff700 1 -- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b1a80 msgr2=0x7fb440158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb437fff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b1a80 0x7fb440158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb437fff700 1 -- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4400ceda0 msgr2=0x7fb440156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb437fff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4400ceda0 0x7fb440156970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb437fff700 1 -- 172.21.15.84:0/3570955363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4300094a0 con 0x7fb4400b81c0 2023-07-06T21:39:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb444f9c700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4400ceda0 0x7fb440156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb43ffff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b1a80 0x7fb440158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb437fff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 0x7fb4400cbde0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb428000c00 tx=0x7fb428004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4280036a0 con 0x7fb4400b81c0 2023-07-06T21:39:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb42801f070 con 0x7fb4400b81c0 2023-07-06T21:39:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4280094a0 con 0x7fb4400b81c0 2023-07-06T21:39:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428003a60 con 0x7fb4400b81c0 2023-07-06T21:39:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.398+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb440157450 con 0x7fb4400b81c0 2023-07-06T21:39:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.399+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb440157970 con 0x7fb4400b81c0 2023-07-06T21:39:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.400+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb428003bc0 con 0x7fb4400b81c0 2023-07-06T21:39:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.402+0000 7fb43d7fa700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb42006ace0 0x7fb42006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.402+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fb428021070 con 0x7fb4400b81c0 2023-07-06T21:39:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.402+0000 7fb43ffff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb42006ace0 0x7fb42006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.402+0000 7fb43ffff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb42006ace0 0x7fb42006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4400ccf70 tx=0x7fb430008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:23.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.406+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb440157970 con 0x7fb4400b81c0 2023-07-06T21:39:23.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.600+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb440157970 con 0x7fb4400b81c0 2023-07-06T21:39:23.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.601+0000 7fb43d7fa700 1 -- 172.21.15.84:0/3570955363 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb440157970 con 0x7fb4400b81c0 2023-07-06T21:39:23.603 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:39:23.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb42006ace0 msgr2=0x7fb42006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb42006ace0 0x7fb42006d190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb4400ccf70 tx=0x7fb430008040 comp rx=0 tx=0).stop 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 msgr2=0x7fb4400cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 0x7fb4400cbde0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb428000c00 tx=0x7fb428004750 comp rx=0 tx=0).stop 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 shutdown_connections 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb42006ace0 0x7fb42006d190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4400ceda0 0x7fb440156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4400b1a80 0x7fb440158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 --2- 172.21.15.84:0/3570955363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4400b81c0 0x7fb4400cbde0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 >> 172.21.15.84:0/3570955363 conn(0x7fb44006b2b0 msgr2=0x7fb44006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:23.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 shutdown_connections 2023-07-06T21:39:23.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.603+0000 7fb4469ff700 1 -- 172.21.15.84:0/3570955363 wait complete. 2023-07-06T21:39:23.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T21:39:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.816+0000 7f75b65ff700 1 Processor -- start 2023-07-06T21:39:23.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.816+0000 7f75b65ff700 1 -- start start 2023-07-06T21:39:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b65ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75b00b83c0 0x7f75b00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b65ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00d3710 0x7f75b00cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b65ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b00cfee0 con 0x7f75b00cfa90 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b65ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b00d0a80 con 0x7f75b00b83c0 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b65ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b00b8790 con 0x7f75b00d3710 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b4b9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f75b00cfa90 0x7f75b015feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52042/0 (socket says 172.21.15.84:52042) 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75b4b9c700 1 -- 172.21.15.84:0/804678117 learned_addr learned my addr 172.21.15.84:0/804678117 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75affff700 1 --2- 172.21.15.84:0/804678117 >> v2:172.21.15.107:3300/0 conn(0x7f75b00b83c0 0x7f75b00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.817+0000 7f75af7fe700 1 --2- 172.21.15.84:0/804678117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00d3710 0x7f75b00cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75aeffd700 1 -- 172.21.15.84:0/804678117 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243351570 0 0) 0x7f75a8001490 con 0x7f75b00cfa90 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75aeffd700 1 -- 172.21.15.84:0/804678117 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75980035b0 con 0x7f75b00cfa90 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75af7fe700 1 -- 172.21.15.84:0/804678117 >> v2:172.21.15.107:3300/0 conn(0x7f75b00b83c0 msgr2=0x7f75b00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75af7fe700 1 --2- 172.21.15.84:0/804678117 >> v2:172.21.15.107:3300/0 conn(0x7f75b00b83c0 0x7f75b00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75af7fe700 1 -- 172.21.15.84:0/804678117 >> v1:172.21.15.84:6789/0 conn(0x7f75b00cfa90 legacy=0x7f75b015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75af7fe700 1 -- 172.21.15.84:0/804678117 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00b4e70 con 0x7f75b00d3710 2023-07-06T21:39:23.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75affff700 1 --2- 172.21.15.84:0/804678117 >> v2:172.21.15.107:3300/0 conn(0x7f75b00b83c0 0x7f75b00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.818+0000 7f75af7fe700 1 --2- 172.21.15.84:0/804678117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00d3710 0x7f75b00cf550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009840 tx=0x7f75a4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251b08a60f3e1b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75aeffd700 1 -- 172.21.15.84:0/804678117 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4011830 con 0x7f75b00d3710 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75aeffd700 1 -- 172.21.15.84:0/804678117 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4011990 con 0x7f75b00d3710 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75aeffd700 1 -- 172.21.15.84:0/804678117 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4011d20 con 0x7f75b00d3710 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 -- 172.21.15.84:0/804678117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00d3710 msgr2=0x7f75b00cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 --2- 172.21.15.84:0/804678117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00d3710 0x7f75b00cf550 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009840 tx=0x7f75a4009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:23.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 -- 172.21.15.84:0/804678117 shutdown_connections 2023-07-06T21:39:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 --2- 172.21.15.84:0/804678117 >> v2:172.21.15.107:3300/0 conn(0x7f75b00b83c0 0x7f75b00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 --2- 172.21.15.84:0/804678117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00d3710 0x7f75b00cf550 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 -- 172.21.15.84:0/804678117 >> 172.21.15.84:0/804678117 conn(0x7f75b006b350 msgr2=0x7f75b006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 -- 172.21.15.84:0/804678117 shutdown_connections 2023-07-06T21:39:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.819+0000 7f75b65ff700 1 -- 172.21.15.84:0/804678117 wait complete. 2023-07-06T21:39:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.821+0000 7f75b65ff700 1 Processor -- start 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.821+0000 7f75b65ff700 1 -- start start 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b65ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 0x7f75b00c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b65ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00cfa90 0x7f75b00cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b65ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b00d3710 0x7f75b015e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b65ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01f2f00 con 0x7f75b00b83c0 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b65ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01f2380 con 0x7f75b00d3710 2023-07-06T21:39:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b65ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b01f2660 con 0x7f75b00cfa90 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75affff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 0x7f75b00c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75affff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 0x7f75b00c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38206/0 (socket says 172.21.15.84:38206) 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75affff700 1 -- 172.21.15.84:0/125906422 learned_addr learned my addr 172.21.15.84:0/125906422 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75b4b9c700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b00d3710 0x7f75b015e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.822+0000 7f75af7fe700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00cfa90 0x7f75b00cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75affff700 1 -- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00cfa90 msgr2=0x7f75b00cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75affff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00cfa90 0x7f75b00cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75affff700 1 -- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b00d3710 msgr2=0x7f75b015e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75affff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b00d3710 0x7f75b015e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75affff700 1 -- 172.21.15.84:0/125906422 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00094a0 con 0x7f75b00b83c0 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75b4b9c700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b00d3710 0x7f75b015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75af7fe700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00cfa90 0x7f75b00cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75affff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 0x7f75b00c6670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75a000ebe0 tx=0x7f75a0009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000c8b0 con 0x7f75b00b83c0 2023-07-06T21:39:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a000ca10 con 0x7f75b00b83c0 2023-07-06T21:39:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a40094a0 con 0x7f75b00b83c0 2023-07-06T21:39:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a000cda0 con 0x7f75b00b83c0 2023-07-06T21:39:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.823+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b01f2c20 con 0x7f75b00b83c0 2023-07-06T21:39:23.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.824+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b00c6bb0 con 0x7f75b00b83c0 2023-07-06T21:39:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.825+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f75a0013470 con 0x7f75b00b83c0 2023-07-06T21:39:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.826+0000 7f75acff9700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f759c06ace0 0x7f759c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.826+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f75a00892e0 con 0x7f75b00b83c0 2023-07-06T21:39:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.826+0000 7f75af7fe700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f759c06ace0 0x7f759c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.827+0000 7f75af7fe700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f759c06ace0 0x7f759c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f75b00d3b20 tx=0x7f75a401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:23.830+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75a00546e0 con 0x7f75b00b83c0 2023-07-06T21:39:24.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.029+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f75b00d2130 con 0x7f75b00b83c0 2023-07-06T21:39:24.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.030+0000 7f75acff9700 1 -- 172.21.15.84:0/125906422 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f75a0057d00 con 0x7f75b00b83c0 2023-07-06T21:39:24.032 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2023-07-06T21:39:24.033 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2023-07-06T21:39:24.034 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:39:24.034 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f759c06ace0 msgr2=0x7f759c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f759c06ace0 0x7f759c06d190 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f75b00d3b20 tx=0x7f75a401c040 comp rx=0 tx=0).stop 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 msgr2=0x7f75b00c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 0x7f75b00c6670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75a000ebe0 tx=0x7f75a0009e20 comp rx=0 tx=0).stop 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 shutdown_connections 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f759c06ace0 0x7f759c06d190 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75b00d3710 0x7f75b015e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75b00b83c0 0x7f75b00c6670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 --2- 172.21.15.84:0/125906422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75b00cfa90 0x7f75b00cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 >> 172.21.15.84:0/125906422 conn(0x7f75b006b350 msgr2=0x7f75b00d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 shutdown_connections 2023-07-06T21:39:24.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.032+0000 7f75b65ff700 1 -- 172.21.15.84:0/125906422 wait complete. 2023-07-06T21:39:24.052 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T21:39:24.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T21:39:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.246+0000 7f5000dff700 1 Processor -- start 2023-07-06T21:39:24.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.246+0000 7f5000dff700 1 -- start start 2023-07-06T21:39:24.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.247+0000 7f5000dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0d0ea0 0x7f4ffc0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.247+0000 7f5000dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4ffc160ac0 0x7f4ffc0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.247+0000 7f5000dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ffc0ceef0 con 0x7f4ffc0cfe20 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.247+0000 7f5000dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ffc0cf450 con 0x7f4ffc160ac0 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.247+0000 7f5000dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0cf120 con 0x7f4ffc0d0ea0 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ffad9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4ffc0cfe20 0x7f4ffc15fee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52052/0 (socket says 172.21.15.84:52052) 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ffad9d700 1 -- 172.21.15.84:0/9684683 learned_addr learned my addr 172.21.15.84:0/9684683 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:24.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1860634711 0 0) 0x7f4ffc0ceef0 con 0x7f4ffc0cfe20 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe80035b0 con 0x7f4ffc0cfe20 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/9684683 >> v2:172.21.15.107:3300/0 conn(0x7f4ffc160ac0 0x7f4ffc0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ffa59c700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0d0ea0 0x7f4ffc0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2976696598 0 0) 0x7f4fe80035b0 con 0x7f4ffc0cfe20 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0d0ea0 msgr2=0x7f4ffc0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0d0ea0 0x7f4ffc0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 >> v2:172.21.15.107:3300/0 conn(0x7f4ffc160ac0 msgr2=0x7f4ffc0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:24.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 --2- 172.21.15.84:0/9684683 >> v2:172.21.15.107:3300/0 conn(0x7f4ffc160ac0 0x7f4ffc0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0cd420 con 0x7f4ffc0cfe20 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.248+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f4ff40019f0 con 0x7f4ffc0cfe20 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 >> v1:172.21.15.84:6789/0 conn(0x7f4ffc0cfe20 legacy=0x7f4ffc15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0cf450 con 0x7f4fe8004850 2023-07-06T21:39:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0cf120 con 0x7f4fe8003a00 2023-07-06T21:39:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800bf50 con 0x7f4fe8005200 2023-07-06T21:39:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ffa59c700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ffad9d700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 -- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe8005200 msgr2=0x7f4fe800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 -- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe8003a00 msgr2=0x7f4fe8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 -- 172.21.15.84:0/9684683 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80035b0 con 0x7f4fe8004850 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.249+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009b60 tx=0x7f4fe8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f40179eb4b3901 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400e6c0 con 0x7f4fe8004850 2023-07-06T21:39:24.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe400e820 con 0x7f4fe8004850 2023-07-06T21:39:24.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f4ff9d9b700 1 -- 172.21.15.84:0/9684683 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe400eb90 con 0x7f4fe8004850 2023-07-06T21:39:24.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f5000dff700 1 -- 172.21.15.84:0/9684683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 msgr2=0x7f4fe8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f5000dff700 1 --2- 172.21.15.84:0/9684683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009b60 tx=0x7f4fe8005880 comp rx=0 tx=0).stop 2023-07-06T21:39:24.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f4ffad9d700 1 -- 172.21.15.84:0/9684683 reap_dead start 2023-07-06T21:39:24.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f5000dff700 1 -- 172.21.15.84:0/9684683 shutdown_connections 2023-07-06T21:39:24.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f5000dff700 1 -- 172.21.15.84:0/9684683 >> 172.21.15.84:0/9684683 conn(0x7f4ffc0715c0 msgr2=0x7f4ffc071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:24.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f5000dff700 1 -- 172.21.15.84:0/9684683 shutdown_connections 2023-07-06T21:39:24.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.250+0000 7f5000dff700 1 -- 172.21.15.84:0/9684683 wait complete. 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.252+0000 7f5000dff700 1 Processor -- start 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.252+0000 7f5000dff700 1 -- start start 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f5000dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4ffc1f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f5000dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0c7a10 0x7f4ffc14d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f5000dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc151b60 0x7f4ffc0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f5000dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c7580 con 0x7f4fe8004850 2023-07-06T21:39:24.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f5000dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc1f5c00 con 0x7f4ffc151b60 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f5000dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ffc0c6970 con 0x7f4ffc0c7a10 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f4ffa59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4ffc1f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f4ffa59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4ffc1f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38222/0 (socket says 172.21.15.84:38222) 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f4ffa59c700 1 -- 172.21.15.84:0/3140734652 learned_addr learned my addr 172.21.15.84:0/3140734652 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0c7a10 0x7f4ffc14d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.253+0000 7f4ffad9d700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc151b60 0x7f4ffc0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffa59c700 1 -- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0c7a10 msgr2=0x7f4ffc14d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffa59c700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0c7a10 0x7f4ffc14d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffa59c700 1 -- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc151b60 msgr2=0x7f4ffc0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffa59c700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc151b60 0x7f4ffc0c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffa59c700 1 -- 172.21.15.84:0/3140734652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc0c6c50 con 0x7f4fe8004850 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffad9d700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc151b60 0x7f4ffc0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ffa59c700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4ffc1f56c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0108c0 tx=0x7f4fec010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec005c20 con 0x7f4fe8004850 2023-07-06T21:39:24.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec00c070 con 0x7f4fe8004850 2023-07-06T21:39:24.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00e660 con 0x7f4fe8004850 2023-07-06T21:39:24.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ffc0c6f30 con 0x7f4fe8004850 2023-07-06T21:39:24.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.254+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ffc14dd40 con 0x7f4fe8004850 2023-07-06T21:39:24.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.255+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ffc04e1d0 con 0x7f4fe8004850 2023-07-06T21:39:24.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.256+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4fec00e7c0 con 0x7f4fe8004850 2023-07-06T21:39:24.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.257+0000 7f4ff2ffd700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fdc06ace0 0x7f4fdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:24.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.257+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4fec089120 con 0x7f4fe8004850 2023-07-06T21:39:24.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.257+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fdc06ace0 0x7f4fdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:24.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.258+0000 7f4ff3fff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fdc06ace0 0x7f4fdc06d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40090b0 tx=0x7f4fe4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:24.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.262+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4fec054520 con 0x7f4fe8004850 2023-07-06T21:39:24.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.458+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4ffc073820 con 0x7f4fe8004850 2023-07-06T21:39:24.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.926+0000 7f4ff2ffd700 1 -- 172.21.15.84:0/3140734652 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4fec057b40 con 0x7f4fe8004850 2023-07-06T21:39:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.931+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fdc06ace0 msgr2=0x7f4fdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.931+0000 7f5000dff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fdc06ace0 0x7f4fdc06d190 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40090b0 tx=0x7f4fe4008030 comp rx=0 tx=0).stop 2023-07-06T21:39:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.931+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 msgr2=0x7f4ffc1f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.931+0000 7f5000dff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4ffc1f56c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0108c0 tx=0x7f4fec010c80 comp rx=0 tx=0).stop 2023-07-06T21:39:24.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 shutdown_connections 2023-07-06T21:39:24.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fdc06ace0 0x7f4fdc06d190 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ffc151b60 0x7f4ffc0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe8004850 0x7f4ffc1f56c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 --2- 172.21.15.84:0/3140734652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ffc0c7a10 0x7f4ffc14d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:24.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 >> 172.21.15.84:0/3140734652 conn(0x7f4ffc0715c0 msgr2=0x7f4ffc15ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:24.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 shutdown_connections 2023-07-06T21:39:24.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:24.932+0000 7f5000dff700 1 -- 172.21.15.84:0/3140734652 wait complete. 2023-07-06T21:39:24.937 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-07-06T21:39:24.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T21:39:25.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.153+0000 7fbb64fff700 1 Processor -- start 2023-07-06T21:39:25.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.153+0000 7fbb64fff700 1 -- start start 2023-07-06T21:39:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb64fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbb60160ac0 0x7fbb600cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb64fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 0x7fbb6015fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb64fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600ceef0 con 0x7fbb600d0ea0 2023-07-06T21:39:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb64fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb600cf450 con 0x7fbb60160ac0 2023-07-06T21:39:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb64fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600cf120 con 0x7fbb600cfe20 2023-07-06T21:39:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb5e59c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbb600d0ea0 0x7fbb600d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52058/0 (socket says 172.21.15.84:52058) 2023-07-06T21:39:25.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.154+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3031731339 learned_addr learned my addr 172.21.15.84:0/3031731339 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:25.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106743457 0 0) 0x7fbb600ceef0 con 0x7fbb600d0ea0 2023-07-06T21:39:25.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5ed9d700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 0x7fbb6015fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5dd9b700 1 --2- 172.21.15.84:0/3031731339 >> v2:172.21.15.107:3300/0 conn(0x7fbb60160ac0 0x7fbb600cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb480035b0 con 0x7fbb600d0ea0 2023-07-06T21:39:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2180036092 0 0) 0x7fbb480035b0 con 0x7fbb600d0ea0 2023-07-06T21:39:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 msgr2=0x7fbb6015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:25.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 0x7fbb6015fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 >> v2:172.21.15.107:3300/0 conn(0x7fbb60160ac0 msgr2=0x7fbb600cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 --2- 172.21.15.84:0/3031731339 >> v2:172.21.15.107:3300/0 conn(0x7fbb60160ac0 0x7fbb600cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600cd420 con 0x7fbb600d0ea0 2023-07-06T21:39:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fbb500017f0 con 0x7fbb600d0ea0 2023-07-06T21:39:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 >> v1:172.21.15.84:6789/0 conn(0x7fbb600d0ea0 legacy=0x7fbb600d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:25.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 0x7fbb48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb48004d90 0x7fbb48005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600cf450 con 0x7fbb48003a00 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600cf120 con 0x7fbb48004d90 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.155+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4800ba90 con 0x7fbb600cfe20 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5ed9d700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb48004d90 0x7fbb48005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5dd9b700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 0x7fbb48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 msgr2=0x7fbb48004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb600cfe20 0x7fbb48004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb48004d90 msgr2=0x7fbb48005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb48004d90 0x7fbb48005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3031731339 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb480035b0 con 0x7fbb48003a00 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c320 tx=0x7fbb48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e68bf12244d8bb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5000ce60 con 0x7fbb48003a00 2023-07-06T21:39:25.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb50007bf0 con 0x7fbb48003a00 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.156+0000 7fbb5d59a700 1 -- 172.21.15.84:0/3031731339 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb500072f0 con 0x7fbb48003a00 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb64fff700 1 -- 172.21.15.84:0/3031731339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 msgr2=0x7fbb48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3031731339 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb48003e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c320 tx=0x7fbb48005880 comp rx=0 tx=0).stop 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb5ed9d700 1 -- 172.21.15.84:0/3031731339 reap_dead start 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb64fff700 1 -- 172.21.15.84:0/3031731339 shutdown_connections 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb64fff700 1 -- 172.21.15.84:0/3031731339 >> 172.21.15.84:0/3031731339 conn(0x7fbb600715c0 msgr2=0x7fbb60071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:25.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb64fff700 1 -- 172.21.15.84:0/3031731339 shutdown_connections 2023-07-06T21:39:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.157+0000 7fbb64fff700 1 -- 172.21.15.84:0/3031731339 wait complete. 2023-07-06T21:39:25.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.159+0000 7fbb64fff700 1 Processor -- start 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.159+0000 7fbb64fff700 1 -- start start 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb64fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb60157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb64fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb60159820 0x7fbb6014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb64fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb601587b0 0x7fbb60150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb64fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601575c0 con 0x7fbb48003a00 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb64fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6014c710 con 0x7fbb601587b0 2023-07-06T21:39:25.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb64fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60156fb0 con 0x7fbb60159820 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb5e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb60157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb5e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb60157a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38248/0 (socket says 172.21.15.84:38248) 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.160+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3656442813 learned_addr learned my addr 172.21.15.84:0/3656442813 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5dd9b700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb60159820 0x7fbb6014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5ed9d700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb601587b0 0x7fbb60150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb60159820 msgr2=0x7fbb6014c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:25.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb60159820 0x7fbb6014c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb601587b0 msgr2=0x7fbb60150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb601587b0 0x7fbb60150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5e59c700 1 -- 172.21.15.84:0/3656442813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb60159120 con 0x7fbb48003a00 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5ed9d700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb601587b0 0x7fbb60150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb5e59c700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb60157a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb50015040 tx=0x7fbb50001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb500072d0 con 0x7fbb48003a00 2023-07-06T21:39:25.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb50007430 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb500077e0 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb60159400 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.161+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb60158c30 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.162+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb601578a0 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.164+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbb50000f70 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.164+0000 7fbb4effd700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbb4006ace0 0x7fbb4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.164+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fbb5008a7b0 con 0x7fbb48003a00 2023-07-06T21:39:25.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.165+0000 7fbb5dd9b700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbb4006ace0 0x7fbb4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:25.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.165+0000 7fbb5dd9b700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbb4006ace0 0x7fbb4006d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb60159bf0 tx=0x7fbb54009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:25.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.169+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbb50055b50 con 0x7fbb48003a00 2023-07-06T21:39:25.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.366+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbb60159590 con 0x7fbb48003a00 2023-07-06T21:39:25.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.931+0000 7fbb4effd700 1 -- 172.21.15.84:0/3656442813 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbb50059170 con 0x7fbb48003a00 2023-07-06T21:39:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.937+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbb4006ace0 msgr2=0x7fbb4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:25.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.937+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbb4006ace0 0x7fbb4006d190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb60159bf0 tx=0x7fbb54009210 comp rx=0 tx=0).stop 2023-07-06T21:39:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 msgr2=0x7fbb60157a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb60157a50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbb50015040 tx=0x7fbb50001bf0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 shutdown_connections 2023-07-06T21:39:25.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbb4006ace0 0x7fbb4006d190 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbb60159bf0 tx=0x7fbb54009210 comp rx=0 tx=0).stop 2023-07-06T21:39:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb601587b0 0x7fbb60150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb48003a00 0x7fbb60157a50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 --2- 172.21.15.84:0/3656442813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb60159820 0x7fbb6014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:25.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.938+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 >> 172.21.15.84:0/3656442813 conn(0x7fbb600715c0 msgr2=0x7fbb600d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.945+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 shutdown_connections 2023-07-06T21:39:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:25.945+0000 7fbb64fff700 1 -- 172.21.15.84:0/3656442813 wait complete. 2023-07-06T21:39:25.957 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-07-06T21:39:25.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T21:39:26.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.170+0000 7faebdfff700 1 Processor -- start 2023-07-06T21:39:26.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.171+0000 7faebdfff700 1 -- start start 2023-07-06T21:39:26.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faebdfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faeb80b81c0 0x7faeb80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faebdfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb80cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faebdfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb80d1830 con 0x7faeb80b1a80 2023-07-06T21:39:26.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faebdfff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faeb80cf330 con 0x7faeb80b81c0 2023-07-06T21:39:26.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faebdfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb80d1270 con 0x7faeb80ceda0 2023-07-06T21:39:26.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb77fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7faeb80b81c0 0x7faeb80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeaffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7faeb80b1a80 0x7faeb815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52062/0 (socket says 172.21.15.84:52062) 2023-07-06T21:39:26.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeaffff700 1 -- 172.21.15.84:0/1954071660 learned_addr learned my addr 172.21.15.84:0/1954071660 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:26.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb7fff700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb80cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb77fe700 1 -- 172.21.15.84:0/1954071660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 msgr2=0x7faeb80cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb77fe700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb80cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb77fe700 1 -- 172.21.15.84:0/1954071660 >> v1:172.21.15.84:6789/0 conn(0x7faeb80b1a80 legacy=0x7faeb815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb77fe700 1 -- 172.21.15.84:0/1954071660 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80d1550 con 0x7faeb80b81c0 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.172+0000 7faeb7fff700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb80cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faeb77fe700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b81c0 0x7faeb80cf600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faea8009840 tx=0x7faea8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a91210091f1612 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faeb6ffd700 1 -- 172.21.15.84:0/1954071660 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8011830 con 0x7faeb80b81c0 2023-07-06T21:39:26.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faeb6ffd700 1 -- 172.21.15.84:0/1954071660 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8011990 con 0x7faeb80b81c0 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faeb6ffd700 1 -- 172.21.15.84:0/1954071660 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8011d20 con 0x7faeb80b81c0 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faebdfff700 1 -- 172.21.15.84:0/1954071660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b81c0 msgr2=0x7faeb80cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faebdfff700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b81c0 0x7faeb80cf600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faea8009840 tx=0x7faea8009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faebdfff700 1 -- 172.21.15.84:0/1954071660 shutdown_connections 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faebdfff700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb80cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faebdfff700 1 --2- 172.21.15.84:0/1954071660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b81c0 0x7faeb80cf600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.173+0000 7faebdfff700 1 -- 172.21.15.84:0/1954071660 >> 172.21.15.84:0/1954071660 conn(0x7faeb806b2b0 msgr2=0x7faeb806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:26.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.174+0000 7faebdfff700 1 -- 172.21.15.84:0/1954071660 shutdown_connections 2023-07-06T21:39:26.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.174+0000 7faebdfff700 1 -- 172.21.15.84:0/1954071660 wait complete. 2023-07-06T21:39:26.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.175+0000 7faebdfff700 1 Processor -- start 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.175+0000 7faebdfff700 1 -- start start 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faebdfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 0x7faeb814b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faebdfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb80b81c0 0x7faeb81e31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faebdfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb81e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faebdfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb814b5c0 con 0x7faeb80b1a80 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faebdfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb81e2d20 con 0x7faeb80b81c0 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faebdfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb81e2110 con 0x7faeb80ceda0 2023-07-06T21:39:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.176+0000 7faeb77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 0x7faeb814b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 0x7faeb814b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38272/0 (socket says 172.21.15.84:38272) 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 -- 172.21.15.84:0/25991116 learned_addr learned my addr 172.21.15.84:0/25991116 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb7fff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb81e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeaffff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb80b81c0 0x7faeb81e31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 -- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 msgr2=0x7faeb81e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb81e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 -- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb80b81c0 msgr2=0x7faeb81e31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb80b81c0 0x7faeb81e31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 -- 172.21.15.84:0/25991116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80094a0 con 0x7faeb80b1a80 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeaffff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb80b81c0 0x7faeb81e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb77fe700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 0x7faeb814b080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faea80090b0 tx=0x7faea8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80038a0 con 0x7faeb80b1a80 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea801f070 con 0x7faeb80b1a80 2023-07-06T21:39:26.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8003c60 con 0x7faeb80b1a80 2023-07-06T21:39:26.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.177+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb81e23f0 con 0x7faeb80b1a80 2023-07-06T21:39:26.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.178+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb81e29b0 con 0x7faeb80b1a80 2023-07-06T21:39:26.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.178+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb804e1d0 con 0x7faeb80b1a80 2023-07-06T21:39:26.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.180+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7faea8026480 con 0x7faeb80b1a80 2023-07-06T21:39:26.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.180+0000 7faeb4ff9700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fae9806ace0 0x7fae9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.181+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5962+0+0 (secure 0 0 0) 0x7faea8021070 con 0x7faeb80b1a80 2023-07-06T21:39:26.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.181+0000 7faeaffff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fae9806ace0 0x7fae9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.181+0000 7faeaffff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fae9806ace0 0x7fae9806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faea0007fd0 tx=0x7faea0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:26.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.184+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7faea8056240 con 0x7faeb80b1a80 2023-07-06T21:39:26.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.425+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faeb81e2b40 con 0x7faeb80b1a80 2023-07-06T21:39:26.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.433+0000 7faeb4ff9700 1 -- 172.21.15.84:0/25991116 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7faea80150c0 con 0x7faeb80b1a80 2023-07-06T21:39:26.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fae9806ace0 msgr2=0x7fae9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fae9806ace0 0x7fae9806d190 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faea0007fd0 tx=0x7faea0009040 comp rx=0 tx=0).stop 2023-07-06T21:39:26.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 msgr2=0x7faeb814b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 0x7faeb814b080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faea80090b0 tx=0x7faea8004750 comp rx=0 tx=0).stop 2023-07-06T21:39:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 shutdown_connections 2023-07-06T21:39:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fae9806ace0 0x7fae9806d190 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faeb80ceda0 0x7faeb81e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faeb80b1a80 0x7faeb814b080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 --2- 172.21.15.84:0/25991116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faeb80b81c0 0x7faeb81e31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.438+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 >> 172.21.15.84:0/25991116 conn(0x7faeb806b2b0 msgr2=0x7faeb80d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.439+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 shutdown_connections 2023-07-06T21:39:26.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.439+0000 7faebdfff700 1 -- 172.21.15.84:0/25991116 wait complete. 2023-07-06T21:39:26.453 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 3 and data pool 4 2023-07-06T21:39:26.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:26.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.666+0000 7fb840fff700 1 Processor -- start 2023-07-06T21:39:26.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.666+0000 7fb840fff700 1 -- start start 2023-07-06T21:39:26.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb840fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c0b83c0 0x7fb83c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb840fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb83c0cfc60 0x7fb83c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb840fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83c0d0a80 con 0x7fb83c0d3710 2023-07-06T21:39:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb840fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83c0b8790 con 0x7fb83c0cfc60 2023-07-06T21:39:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb840fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c0b4e70 con 0x7fb83c0b83c0 2023-07-06T21:39:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb839d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb83c0d3710 0x7fb83c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52064/0 (socket says 172.21.15.84:52064) 2023-07-06T21:39:26.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb839d9b700 1 -- 172.21.15.84:0/2547578763 learned_addr learned my addr 172.21.15.84:0/2547578763 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/2547578763 >> v2:172.21.15.107:3300/0 conn(0x7fb83c0cfc60 0x7fb83c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.667+0000 7fb83a59c700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c0b83c0 0x7fb83c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341212089 0 0) 0x7fb83c0d0a80 con 0x7fb83c0d3710 2023-07-06T21:39:26.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8240035b0 con 0x7fb83c0d3710 2023-07-06T21:39:26.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4126416081 0 0) 0x7fb8240035b0 con 0x7fb83c0d3710 2023-07-06T21:39:26.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c0b83c0 msgr2=0x7fb83c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:26.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c0b83c0 0x7fb83c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 >> v2:172.21.15.107:3300/0 conn(0x7fb83c0cfc60 msgr2=0x7fb83c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:26.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 --2- 172.21.15.84:0/2547578763 >> v2:172.21.15.107:3300/0 conn(0x7fb83c0cfc60 0x7fb83c0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c0b5150 con 0x7fb83c0d3710 2023-07-06T21:39:26.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fb830001670 con 0x7fb83c0d3710 2023-07-06T21:39:26.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 >> v1:172.21.15.84:6789/0 conn(0x7fb83c0d3710 legacy=0x7fb83c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb824003a00 0x7fb824003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8240043e0 0x7fb8240048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb824004fc0 0x7fb82400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c0b8790 con 0x7fb824004fc0 2023-07-06T21:39:26.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c0b4e70 con 0x7fb8240043e0 2023-07-06T21:39:26.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82400bf50 con 0x7fb824003a00 2023-07-06T21:39:26.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/2547578763 >> v2:172.21.15.107:3300/0 conn(0x7fb83c0cfc60 0x7fb83c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:26.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83a59c700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c0b83c0 0x7fb83c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:26.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb839d9b700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8240043e0 0x7fb8240048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.668+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb824004fc0 0x7fb82400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83a59c700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb824003a00 0x7fb824003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83ad9d700 1 -- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb824003a00 msgr2=0x7fb824003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:26.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb824003a00 0x7fb824003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83ad9d700 1 -- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8240043e0 msgr2=0x7fb8240048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:26.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8240043e0 0x7fb8240048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83ad9d700 1 -- 172.21.15.84:0/2547578763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8240035b0 con 0x7fb824004fc0 2023-07-06T21:39:26.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb839d9b700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8240043e0 0x7fb8240048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:26.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb824004fc0 0x7fb82400ba10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e350 tx=0x7fb824005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b948b6e1c7100c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:26.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400ee10 con 0x7fb824004fc0 2023-07-06T21:39:26.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83400f040 con 0x7fb824004fc0 2023-07-06T21:39:26.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.669+0000 7fb83959a700 1 -- 172.21.15.84:0/2547578763 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834003c40 con 0x7fb824004fc0 2023-07-06T21:39:26.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb840fff700 1 -- 172.21.15.84:0/2547578763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb824004fc0 msgr2=0x7fb82400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb840fff700 1 --2- 172.21.15.84:0/2547578763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb824004fc0 0x7fb82400ba10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb83400e350 tx=0x7fb824005880 comp rx=0 tx=0).stop 2023-07-06T21:39:26.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb83ad9d700 1 -- 172.21.15.84:0/2547578763 reap_dead start 2023-07-06T21:39:26.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb840fff700 1 -- 172.21.15.84:0/2547578763 shutdown_connections 2023-07-06T21:39:26.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb840fff700 1 -- 172.21.15.84:0/2547578763 >> 172.21.15.84:0/2547578763 conn(0x7fb83c06b350 msgr2=0x7fb83c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:26.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb840fff700 1 -- 172.21.15.84:0/2547578763 shutdown_connections 2023-07-06T21:39:26.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.670+0000 7fb840fff700 1 -- 172.21.15.84:0/2547578763 wait complete. 2023-07-06T21:39:26.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.672+0000 7fb840fff700 1 Processor -- start 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.672+0000 7fb840fff700 1 -- start start 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb840fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 0x7fb83c0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb840fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb83c0caf00 0x7fb83c0cb370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb840fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c1f46e0 0x7fb83c1f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb840fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c1f6290 con 0x7fb83c0caf00 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb840fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c1f5090 con 0x7fb824004fc0 2023-07-06T21:39:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb840fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb83c1f5680 con 0x7fb83c1f46e0 2023-07-06T21:39:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb83a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 0x7fb83c0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb839d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb83c0caf00 0x7fb83c0cb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb83a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 0x7fb83c0b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:57528/0 (socket says 172.21.15.84:57528) 2023-07-06T21:39:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb83a59c700 1 -- 172.21.15.84:0/1502721797 learned_addr learned my addr 172.21.15.84:0/1502721797 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.673+0000 7fb83ad9d700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c1f46e0 0x7fb83c1f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb83a59c700 1 -- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c1f46e0 msgr2=0x7fb83c1f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb83a59c700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c1f46e0 0x7fb83c1f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb83a59c700 1 -- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb83c0caf00 msgr2=0x7fb83c0cb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb83a59c700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb83c0caf00 0x7fb83c0cb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb83a59c700 1 -- 172.21.15.84:0/1502721797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c1f5960 con 0x7fb824004fc0 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb839d9b700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb83c0caf00 0x7fb83c0cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb83a59c700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 0x7fb83c0b6c30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb82c011c30 tx=0x7fb82c011f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00fa00 con 0x7fb824004fc0 2023-07-06T21:39:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c00fb60 con 0x7fb824004fc0 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00fed0 con 0x7fb824004fc0 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83c1f5c40 con 0x7fb824004fc0 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.674+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c1f52a0 con 0x7fb824004fc0 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.675+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83c15b260 con 0x7fb824004fc0 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.676+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb82c014440 con 0x7fb824004fc0 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.677+0000 7fb82affd700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb81c06ace0 0x7fb81c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:26.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.677+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb82c089c70 con 0x7fb824004fc0 2023-07-06T21:39:26.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.677+0000 7fb839d9b700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb81c06ace0 0x7fb81c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:26.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.678+0000 7fb839d9b700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb81c06ace0 0x7fb81c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb830001050 tx=0x7fb830007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:26.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.683+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb82c054d20 con 0x7fb824004fc0 2023-07-06T21:39:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.883+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb83c1f5430 con 0x7fb824004fc0 2023-07-06T21:39:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.885+0000 7fb82affd700 1 -- 172.21.15.84:0/1502721797 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18292 (secure 0 0 0) 0x7fb82c022090 con 0x7fb824004fc0 2023-07-06T21:39:26.889 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:26.889 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":22,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:26.426994+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:26.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb81c06ace0 msgr2=0x7fb81c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb81c06ace0 0x7fb81c06d190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb830001050 tx=0x7fb830007980 comp rx=0 tx=0).stop 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 msgr2=0x7fb83c0b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 0x7fb83c0b6c30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb82c011c30 tx=0x7fb82c011f40 comp rx=0 tx=0).stop 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 shutdown_connections 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb81c06ace0 0x7fb81c06d190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb83c1f46e0 0x7fb83c1f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb824004fc0 0x7fb83c0b6c30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 --2- 172.21.15.84:0/1502721797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb83c0caf00 0x7fb83c0cb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 >> 172.21.15.84:0/1502721797 conn(0x7fb83c06b350 msgr2=0x7fb83c0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 shutdown_connections 2023-07-06T21:39:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:26.887+0000 7fb840fff700 1 -- 172.21.15.84:0/1502721797 wait complete. 2023-07-06T21:39:26.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:27.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.100+0000 7f43e1bff700 1 Processor -- start 2023-07-06T21:39:27.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.100+0000 7f43e1bff700 1 -- start start 2023-07-06T21:39:27.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.101+0000 7f43e1bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43dc0b83c0 0x7f43dc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.101+0000 7f43e1bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f43dc0d3710 0x7f43dc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.101+0000 7f43e1bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0cfa90 0x7f43dc15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.101+0000 7f43e1bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0cff10 con 0x7f43dc0b83c0 2023-07-06T21:39:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.101+0000 7f43e1bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0d0a80 con 0x7f43dc0d3710 2023-07-06T21:39:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.101+0000 7f43e1bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0b8790 con 0x7f43dc0cfa90 2023-07-06T21:39:27.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43dc0b83c0 0x7f43dc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f43dc0b83c0 0x7f43dc0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38300/0 (socket says 172.21.15.84:38300) 2023-07-06T21:39:27.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 -- 172.21.15.84:0/4190619506 learned_addr learned my addr 172.21.15.84:0/4190619506 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:27.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43dbfff700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0cfa90 0x7f43dc15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43daffd700 1 --2- 172.21.15.84:0/4190619506 >> v2:172.21.15.107:3300/0 conn(0x7f43dc0d3710 0x7f43dc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 -- 172.21.15.84:0/4190619506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0cfa90 msgr2=0x7f43dc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0cfa90 0x7f43dc15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 -- 172.21.15.84:0/4190619506 >> v2:172.21.15.107:3300/0 conn(0x7f43dc0d3710 msgr2=0x7f43dc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 --2- 172.21.15.84:0/4190619506 >> v2:172.21.15.107:3300/0 conn(0x7f43dc0d3710 0x7f43dc0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 -- 172.21.15.84:0/4190619506 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0b4e70 con 0x7f43dc0b83c0 2023-07-06T21:39:27.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43dbfff700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0cfa90 0x7f43dc15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43daffd700 1 --2- 172.21.15.84:0/4190619506 >> v2:172.21.15.107:3300/0 conn(0x7f43dc0d3710 0x7f43dc0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.102+0000 7f43db7fe700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0cd200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0b8cb0 tx=0x7f43cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea8d505c1d31a9e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43da7fc700 1 -- 172.21.15.84:0/4190619506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00b870 con 0x7f43dc0b83c0 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43da7fc700 1 -- 172.21.15.84:0/4190619506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc00b9d0 con 0x7f43dc0b83c0 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43da7fc700 1 -- 172.21.15.84:0/4190619506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00bd60 con 0x7f43dc0b83c0 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 -- 172.21.15.84:0/4190619506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 msgr2=0x7f43dc0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0cd200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0b8cb0 tx=0x7f43cc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 -- 172.21.15.84:0/4190619506 shutdown_connections 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0cfa90 0x7f43dc15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 --2- 172.21.15.84:0/4190619506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0cd200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 --2- 172.21.15.84:0/4190619506 >> v2:172.21.15.107:3300/0 conn(0x7f43dc0d3710 0x7f43dc0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 -- 172.21.15.84:0/4190619506 >> 172.21.15.84:0/4190619506 conn(0x7f43dc06b350 msgr2=0x7f43dc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 -- 172.21.15.84:0/4190619506 shutdown_connections 2023-07-06T21:39:27.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.103+0000 7f43e1bff700 1 -- 172.21.15.84:0/4190619506 wait complete. 2023-07-06T21:39:27.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.105+0000 7f43e1bff700 1 Processor -- start 2023-07-06T21:39:27.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.105+0000 7f43e1bff700 1 -- start start 2023-07-06T21:39:27.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43e1bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43e1bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc0cfa90 0x7f43dc153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43e1bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0d3710 0x7f43dc157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43e1bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc156f60 con 0x7f43dc0b83c0 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43e1bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc1562e0 con 0x7f43dc0cfa90 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43e1bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc1565c0 con 0x7f43dc0d3710 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0c90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38316/0 (socket says 172.21.15.84:38316) 2023-07-06T21:39:27.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.106+0000 7f43db7fe700 1 -- 172.21.15.84:0/3958281936 learned_addr learned my addr 172.21.15.84:0/3958281936 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43daffd700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc0cfa90 0x7f43dc153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43dbfff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0d3710 0x7f43dc157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43db7fe700 1 -- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0d3710 msgr2=0x7f43dc157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43db7fe700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0d3710 0x7f43dc157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43db7fe700 1 -- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc0cfa90 msgr2=0x7f43dc153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43db7fe700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc0cfa90 0x7f43dc153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43db7fe700 1 -- 172.21.15.84:0/3958281936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009550 con 0x7f43dc0b83c0 2023-07-06T21:39:27.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43daffd700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc0cfa90 0x7f43dc153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43db7fe700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0c90d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f43cc014040 tx=0x7f43cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc004520 con 0x7f43dc0b83c0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc01e070 con 0x7f43dc0b83c0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0038e0 con 0x7f43dc0b83c0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc1568a0 con 0x7f43dc0b83c0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.107+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc1544a0 con 0x7f43dc0b83c0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.108+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc15ad90 con 0x7f43dc0b83c0 2023-07-06T21:39:27.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.109+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43cc003a40 con 0x7f43dc0b83c0 2023-07-06T21:39:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.110+0000 7f43c3fff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43c806ad10 0x7f43c806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.110+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f43cc020070 con 0x7f43dc0b83c0 2023-07-06T21:39:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.110+0000 7f43daffd700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43c806ad10 0x7f43c806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.111+0000 7f43daffd700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43c806ad10 0x7f43c806d1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006fd0 tx=0x7f43d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:27.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.115+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f43cc055b80 con 0x7f43dc0b83c0 2023-07-06T21:39:27.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.313+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43dc15aab0 con 0x7f43dc0b83c0 2023-07-06T21:39:27.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.315+0000 7f43c3fff700 1 -- 172.21.15.84:0/3958281936 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+18292 (secure 0 0 0) 0x7f43cc015090 con 0x7f43dc0b83c0 2023-07-06T21:39:27.318 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:27.319 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":22,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:26.426994+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:27.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43c806ad10 msgr2=0x7f43c806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43c806ad10 0x7f43c806d1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006fd0 tx=0x7f43d0008040 comp rx=0 tx=0).stop 2023-07-06T21:39:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 msgr2=0x7f43dc0c90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0c90d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f43cc014040 tx=0x7f43cc004750 comp rx=0 tx=0).stop 2023-07-06T21:39:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 shutdown_connections 2023-07-06T21:39:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43c806ad10 0x7f43c806d1c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc0d3710 0x7f43dc157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43dc0b83c0 0x7f43dc0c90d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 --2- 172.21.15.84:0/3958281936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc0cfa90 0x7f43dc153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 >> 172.21.15.84:0/3958281936 conn(0x7f43dc06b350 msgr2=0x7f43dc15bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:27.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 shutdown_connections 2023-07-06T21:39:27.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.317+0000 7f43e1bff700 1 -- 172.21.15.84:0/3958281936 wait complete. 2023-07-06T21:39:27.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T21:39:27.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.535+0000 7fa6105ff700 1 Processor -- start 2023-07-06T21:39:27.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.535+0000 7fa6105ff700 1 -- start start 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.536+0000 7fa6105ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080cfd10 0x7fa6080d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.536+0000 7fa6105ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6080b1a80 0x7fa6080d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.536+0000 7fa6105ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6080d1830 con 0x7fa6080b81c0 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.536+0000 7fa6105ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6080cf330 con 0x7fa6080b1a80 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.536+0000 7fa6105ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6080d1270 con 0x7fa6080cfd10 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6080b1a80 0x7fa6080d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa6080b1a80 0x7fa6080d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38328/0 (socket says 172.21.15.84:38328) 2023-07-06T21:39:27.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 -- 172.21.15.84:0/3913558810 learned_addr learned my addr 172.21.15.84:0/3913558810 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:27.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60db9a700 1 --2- 172.21.15.84:0/3913558810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080cfd10 0x7fa6080d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 -- 172.21.15.84:0/3913558810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080cfd10 msgr2=0x7fa6080d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:27.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 --2- 172.21.15.84:0/3913558810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080cfd10 0x7fa6080d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 -- 172.21.15.84:0/3913558810 >> v1:172.21.15.107:6789/0 conn(0x7fa6080b81c0 legacy=0x7fa6080cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 -- 172.21.15.84:0/3913558810 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6080d1550 con 0x7fa6080b1a80 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.537+0000 7fa60eb9c700 1 --2- 172.21.15.84:0/3913558810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa6080d28c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa600009e80 tx=0x7fa60000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed89b216200b621 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa60d399700 1 -- 172.21.15.84:0/3913558810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000e070 con 0x7fa6080b1a80 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa60d399700 1 -- 172.21.15.84:0/3913558810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60000b8d0 con 0x7fa6080b1a80 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa60d399700 1 -- 172.21.15.84:0/3913558810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60000bc80 con 0x7fa6080b1a80 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 -- 172.21.15.84:0/3913558810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 msgr2=0x7fa6080d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 --2- 172.21.15.84:0/3913558810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa6080d28c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa600009e80 tx=0x7fa60000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 -- 172.21.15.84:0/3913558810 shutdown_connections 2023-07-06T21:39:27.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 --2- 172.21.15.84:0/3913558810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa6080d28c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 --2- 172.21.15.84:0/3913558810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080cfd10 0x7fa6080d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 -- 172.21.15.84:0/3913558810 >> 172.21.15.84:0/3913558810 conn(0x7fa60806b2b0 msgr2=0x7fa60806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:27.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 -- 172.21.15.84:0/3913558810 shutdown_connections 2023-07-06T21:39:27.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.538+0000 7fa6105ff700 1 -- 172.21.15.84:0/3913558810 wait complete. 2023-07-06T21:39:27.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.540+0000 7fa6105ff700 1 Processor -- start 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.540+0000 7fa6105ff700 1 -- start start 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa6105ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa60815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa6105ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080b81c0 0x7fa60815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa6105ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6080cfd10 0x7fa60815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa6105ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60815da60 con 0x7fa6080b1a80 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa6105ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60815d430 con 0x7fa6080cfd10 2023-07-06T21:39:27.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa6105ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60815e3c0 con 0x7fa6080b81c0 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.541+0000 7fa60e39b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa60815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa60815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38344/0 (socket says 172.21.15.84:38344) 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 -- 172.21.15.84:0/2631056155 learned_addr learned my addr 172.21.15.84:0/2631056155 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60eb9c700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6080cfd10 0x7fa60815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 -- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080b81c0 msgr2=0x7fa60815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080b81c0 0x7fa60815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 -- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6080cfd10 msgr2=0x7fa60815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6080cfd10 0x7fa60815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 -- 172.21.15.84:0/2631056155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa600009b30 con 0x7fa6080b1a80 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa60e39b700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa60815de80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c4e0 tx=0x7fa60400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040052d0 con 0x7fa6080b1a80 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa604005430 con 0x7fa6080b1a80 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6040038c0 con 0x7fa6080b1a80 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.542+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6080cc6a0 con 0x7fa6080b1a80 2023-07-06T21:39:27.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.543+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60815d680 con 0x7fa6080b1a80 2023-07-06T21:39:27.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.544+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60815d810 con 0x7fa6080b1a80 2023-07-06T21:39:27.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.544+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa604003a20 con 0x7fa6080b1a80 2023-07-06T21:39:27.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.545+0000 7fa5feffd700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5ec06ace0 0x7fa5ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:27.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.545+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa604088a40 con 0x7fa6080b1a80 2023-07-06T21:39:27.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.545+0000 7fa60db9a700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5ec06ace0 0x7fa5ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:27.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.546+0000 7fa60db9a700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5ec06ace0 0x7fa5ec06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006b20 tx=0x7fa5f8006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:27.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.550+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa604053a50 con 0x7fa6080b1a80 2023-07-06T21:39:27.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:27.793+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa60815dcd0 con 0x7fa6080b1a80 2023-07-06T21:39:28.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.436+0000 7fa5feffd700 1 -- 172.21.15.84:0/2631056155 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa604057070 con 0x7fa6080b1a80 2023-07-06T21:39:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5ec06ace0 msgr2=0x7fa5ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5ec06ace0 0x7fa5ec06d190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006b20 tx=0x7fa5f8006ab0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 msgr2=0x7fa60815de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa60815de80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa60400c4e0 tx=0x7fa60400c8a0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 shutdown_connections 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa5ec06ace0 0x7fa5ec06d190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6080cfd10 0x7fa60815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6080b1a80 0x7fa60815de80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 --2- 172.21.15.84:0/2631056155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6080b81c0 0x7fa60815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 >> 172.21.15.84:0/2631056155 conn(0x7fa60806b2b0 msgr2=0x7fa608073310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 shutdown_connections 2023-07-06T21:39:28.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.439+0000 7fa6105ff700 1 -- 172.21.15.84:0/2631056155 wait complete. 2023-07-06T21:39:28.462 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T21:39:28.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T21:39:28.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.659+0000 7fe56a7ff700 1 Processor -- start 2023-07-06T21:39:28.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.659+0000 7fe56a7ff700 1 -- start start 2023-07-06T21:39:28.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe56a7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe5640b81c0 0x7fe5640cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:28.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe56a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640ceda0 0x7fe5640cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe56a7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5640d1830 con 0x7fe5640b1a80 2023-07-06T21:39:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe56a7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5640cf330 con 0x7fe5640b81c0 2023-07-06T21:39:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe56a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640d1270 con 0x7fe5640ceda0 2023-07-06T21:39:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe563fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe5640b81c0 0x7fe5640cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe568d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640ceda0 0x7fe5640cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:28.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe568d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640ceda0 0x7fe5640cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:39312/0 (socket says 172.21.15.84:39312) 2023-07-06T21:39:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe568d9c700 1 -- 172.21.15.84:0/4151494695 learned_addr learned my addr 172.21.15.84:0/4151494695 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe563fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe5640b81c0 0x7fe5640cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:57570/0 (socket says 172.21.15.84:57570) 2023-07-06T21:39:28.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.660+0000 7fe55bfff700 1 --1- 172.21.15.84:0/4151494695 >> v1:172.21.15.107:6789/0 conn(0x7fe5640b1a80 0x7fe56415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:37026/0 (socket says 172.21.15.84:37026) 2023-07-06T21:39:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe5637fe700 1 -- 172.21.15.84:0/4151494695 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482906352 0 0) 0x7fe5640d1830 con 0x7fe5640b1a80 2023-07-06T21:39:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe563fff700 1 -- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640ceda0 msgr2=0x7fe5640cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe563fff700 1 --2- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640ceda0 0x7fe5640cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe563fff700 1 -- 172.21.15.84:0/4151494695 >> v1:172.21.15.107:6789/0 conn(0x7fe5640b1a80 legacy=0x7fe56415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe563fff700 1 -- 172.21.15.84:0/4151494695 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5640d1550 con 0x7fe5640b81c0 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe563fff700 1 --2- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640b81c0 0x7fe5640cf600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe554009840 tx=0x7fe554009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=201a5118bf1b9ecf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe5637fe700 1 -- 172.21.15.84:0/4151494695 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554011830 con 0x7fe5640b81c0 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe5637fe700 1 -- 172.21.15.84:0/4151494695 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554011990 con 0x7fe5640b81c0 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.661+0000 7fe5637fe700 1 -- 172.21.15.84:0/4151494695 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554011d20 con 0x7fe5640b81c0 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 -- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640b81c0 msgr2=0x7fe5640cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:28.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 --2- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640b81c0 0x7fe5640cf600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe554009840 tx=0x7fe554009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:28.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 -- 172.21.15.84:0/4151494695 shutdown_connections 2023-07-06T21:39:28.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 --2- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640ceda0 0x7fe5640cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 --2- 172.21.15.84:0/4151494695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640b81c0 0x7fe5640cf600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 -- 172.21.15.84:0/4151494695 >> 172.21.15.84:0/4151494695 conn(0x7fe56406b2b0 msgr2=0x7fe56406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:28.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 -- 172.21.15.84:0/4151494695 shutdown_connections 2023-07-06T21:39:28.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.662+0000 7fe56a7ff700 1 -- 172.21.15.84:0/4151494695 wait complete. 2023-07-06T21:39:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.664+0000 7fe56a7ff700 1 Processor -- start 2023-07-06T21:39:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.664+0000 7fe56a7ff700 1 -- start start 2023-07-06T21:39:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe56a7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 0x7fe564156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:28.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe56a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640b81c0 0x7fe564155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe56a7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640ceda0 0x7fe5641593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe56a7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640cb160 con 0x7fe5640b1a80 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe56a7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564157fc0 con 0x7fe5640ceda0 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe56a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640cae50 con 0x7fe5640b81c0 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe563fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 0x7fe564156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe563fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 0x7fe564156a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38348/0 (socket says 172.21.15.84:38348) 2023-07-06T21:39:28.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.665+0000 7fe563fff700 1 -- 172.21.15.84:0/204516796 learned_addr learned my addr 172.21.15.84:0/204516796 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe563fff700 1 -- 172.21.15.84:0/204516796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640b81c0 msgr2=0x7fe564155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe563fff700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640b81c0 0x7fe564155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe563fff700 1 -- 172.21.15.84:0/204516796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640ceda0 msgr2=0x7fe5641593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe563fff700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640ceda0 0x7fe5641593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe563fff700 1 -- 172.21.15.84:0/204516796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540094a0 con 0x7fe5640b1a80 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe563fff700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 0x7fe564156a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe554009fd0 tx=0x7fe554004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:28.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540036a0 con 0x7fe5640b1a80 2023-07-06T21:39:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55401f070 con 0x7fe5640b1a80 2023-07-06T21:39:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554003a60 con 0x7fe5640b1a80 2023-07-06T21:39:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe56a7ff700 1 -- 172.21.15.84:0/204516796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe564157440 con 0x7fe5640b1a80 2023-07-06T21:39:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.666+0000 7fe56a7ff700 1 -- 172.21.15.84:0/204516796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe564157a00 con 0x7fe5640b1a80 2023-07-06T21:39:28.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.667+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5640cb2a0 con 0x7fe5640b1a80 2023-07-06T21:39:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.668+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe554003bc0 con 0x7fe5640b1a80 2023-07-06T21:39:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.669+0000 7fe5617fa700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe54406ace0 0x7fe54406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.669+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe554021070 con 0x7fe5640b1a80 2023-07-06T21:39:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.669+0000 7fe55bfff700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe54406ace0 0x7fe54406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:28.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.670+0000 7fe55bfff700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe54406ace0 0x7fe54406d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5640cab80 tx=0x7fe54c0079f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:28.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.673+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe5640cb2a0 con 0x7fe5640b1a80 2023-07-06T21:39:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:28.913+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe5640cb2a0 con 0x7fe5640b1a80 2023-07-06T21:39:29.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.917+0000 7fe5617fa700 1 -- 172.21.15.84:0/204516796 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe5640cb2a0 con 0x7fe5640b1a80 2023-07-06T21:39:29.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe54406ace0 msgr2=0x7fe54406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:29.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe54406ace0 0x7fe54406d190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe5640cab80 tx=0x7fe54c0079f0 comp rx=0 tx=0).stop 2023-07-06T21:39:29.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 msgr2=0x7fe564156a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:29.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 0x7fe564156a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe554009fd0 tx=0x7fe554004750 comp rx=0 tx=0).stop 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 shutdown_connections 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe54406ace0 0x7fe54406d190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5640ceda0 0x7fe5641593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe5640b1a80 0x7fe564156a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 --2- 172.21.15.84:0/204516796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5640b81c0 0x7fe564155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 >> 172.21.15.84:0/204516796 conn(0x7fe56406b2b0 msgr2=0x7fe5640d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 shutdown_connections 2023-07-06T21:39:29.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:29.921+0000 7fe55a7fc700 1 -- 172.21.15.84:0/204516796 wait complete. 2023-07-06T21:39:29.944 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T21:39:29.945 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:39:30.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.139+0000 7fd0eedff700 1 Processor -- start 2023-07-06T21:39:30.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.139+0000 7fd0eedff700 1 -- start start 2023-07-06T21:39:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0eedff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd0e80d3710 0x7fd0e80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0eedff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80cd910 con 0x7fd0e80cfa90 2023-07-06T21:39:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0eedff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e80d0a80 con 0x7fd0e80b83c0 2023-07-06T21:39:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0eedff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e80cff10 con 0x7fd0e80d3710 2023-07-06T21:39:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0ed39c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd0e80cfa90 0x7fd0e80d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52068/0 (socket says 172.21.15.84:52068) 2023-07-06T21:39:30.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0ed39c700 1 -- 172.21.15.84:0/124887434 learned_addr learned my addr 172.21.15.84:0/124887434 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:30.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e7fff700 1 --2- 172.21.15.84:0/124887434 >> v2:172.21.15.84:3300/0 conn(0x7fd0e80d3710 0x7fd0e80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e77fe700 1 -- 172.21.15.84:0/124887434 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4178589652 0 0) 0x7fd0e0001490 con 0x7fd0e80cfa90 2023-07-06T21:39:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e77fe700 1 -- 172.21.15.84:0/124887434 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d80035b0 con 0x7fd0e80cfa90 2023-07-06T21:39:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e7fff700 1 -- 172.21.15.84:0/124887434 >> v1:172.21.15.107:6789/0 conn(0x7fd0e80b83c0 legacy=0x7fd0e80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e77fe700 1 -- 172.21.15.84:0/124887434 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546451478 0 0) 0x7fd0e80d0a80 con 0x7fd0e80b83c0 2023-07-06T21:39:30.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e7fff700 1 -- 172.21.15.84:0/124887434 >> v1:172.21.15.84:6789/0 conn(0x7fd0e80cfa90 legacy=0x7fd0e80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.141+0000 7fd0e7fff700 1 -- 172.21.15.84:0/124887434 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80b8790 con 0x7fd0e80d3710 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.142+0000 7fd0e7fff700 1 --2- 172.21.15.84:0/124887434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80d3710 0x7fd0e80cf550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009840 tx=0x7fd0d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c396621c7dcb37f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.142+0000 7fd0e77fe700 1 -- 172.21.15.84:0/124887434 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011830 con 0x7fd0e80d3710 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.142+0000 7fd0e77fe700 1 -- 172.21.15.84:0/124887434 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d4011990 con 0x7fd0e80d3710 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.142+0000 7fd0e77fe700 1 -- 172.21.15.84:0/124887434 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4011d20 con 0x7fd0e80d3710 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 -- 172.21.15.84:0/124887434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80d3710 msgr2=0x7fd0e80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 --2- 172.21.15.84:0/124887434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80d3710 0x7fd0e80cf550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009840 tx=0x7fd0d4009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 -- 172.21.15.84:0/124887434 shutdown_connections 2023-07-06T21:39:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 --2- 172.21.15.84:0/124887434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80d3710 0x7fd0e80cf550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 -- 172.21.15.84:0/124887434 >> 172.21.15.84:0/124887434 conn(0x7fd0e806b350 msgr2=0x7fd0e806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 -- 172.21.15.84:0/124887434 shutdown_connections 2023-07-06T21:39:30.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.143+0000 7fd0eedff700 1 -- 172.21.15.84:0/124887434 wait complete. 2023-07-06T21:39:30.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.145+0000 7fd0eedff700 1 Processor -- start 2023-07-06T21:39:30.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.145+0000 7fd0eedff700 1 -- start start 2023-07-06T21:39:30.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0eedff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 0x7fd0e815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0eedff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0e80cfa90 0x7fd0e8150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0eedff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e80d3710 0x7fd0e80c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0eedff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80c88d0 con 0x7fd0e80b83c0 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0eedff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e815dfe0 con 0x7fd0e80d3710 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0eedff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80c7cc0 con 0x7fd0e80cfa90 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 0x7fd0e815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 0x7fd0e815daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38376/0 (socket says 172.21.15.84:38376) 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 -- 172.21.15.84:0/2635405768 learned_addr learned my addr 172.21.15.84:0/2635405768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ed39c700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e80d3710 0x7fd0e80c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 -- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0e80cfa90 msgr2=0x7fd0e8150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0e80cfa90 0x7fd0e8150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 -- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e80d3710 msgr2=0x7fd0e80c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e80d3710 0x7fd0e80c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ecb9b700 1 -- 172.21.15.84:0/2635405768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d40094a0 con 0x7fd0e80b83c0 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.146+0000 7fd0ed39c700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e80d3710 0x7fd0e80c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.147+0000 7fd0ecb9b700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 0x7fd0e815daa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00c550 tx=0x7fd0dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.147+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0054e0 con 0x7fd0e80b83c0 2023-07-06T21:39:30.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.147+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc003680 con 0x7fd0e80b83c0 2023-07-06T21:39:30.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.147+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e80c7fa0 con 0x7fd0e80b83c0 2023-07-06T21:39:30.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.147+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc003a30 con 0x7fd0e80b83c0 2023-07-06T21:39:30.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.147+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80c8560 con 0x7fd0e80b83c0 2023-07-06T21:39:30.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.148+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e80c8bb0 con 0x7fd0e80b83c0 2023-07-06T21:39:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.149+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd0dc003b90 con 0x7fd0e80b83c0 2023-07-06T21:39:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.150+0000 7fd0e57fa700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd0cc06ace0 0x7fd0cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.150+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd0dc0889f0 con 0x7fd0e80b83c0 2023-07-06T21:39:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.150+0000 7fd0e7fff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd0cc06ace0 0x7fd0cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.151+0000 7fd0e7fff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd0cc06ace0 0x7fd0cc06d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80d3b20 tx=0x7fd0d4010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:30.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.155+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd0dc053a00 con 0x7fd0e80b83c0 2023-07-06T21:39:30.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.391+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0e80d2470 con 0x7fd0e80b83c0 2023-07-06T21:39:30.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.391+0000 7fd0e57fa700 1 -- 172.21.15.84:0/2635405768 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7fd0dc017020 con 0x7fd0e80b83c0 2023-07-06T21:39:30.395 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:30.395 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:39:30.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd0cc06ace0 msgr2=0x7fd0cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd0cc06ace0 0x7fd0cc06d190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd0e80d3b20 tx=0x7fd0d4010000 comp rx=0 tx=0).stop 2023-07-06T21:39:30.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 msgr2=0x7fd0e815daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 0x7fd0e815daa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00c550 tx=0x7fd0dc00c910 comp rx=0 tx=0).stop 2023-07-06T21:39:30.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 shutdown_connections 2023-07-06T21:39:30.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd0cc06ace0 0x7fd0cc06d190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0e80d3710 0x7fd0e80c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0e80b83c0 0x7fd0e815daa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 --2- 172.21.15.84:0/2635405768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0e80cfa90 0x7fd0e8150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 >> 172.21.15.84:0/2635405768 conn(0x7fd0e806b350 msgr2=0x7fd0e80d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:30.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 shutdown_connections 2023-07-06T21:39:30.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.394+0000 7fd0eedff700 1 -- 172.21.15.84:0/2635405768 wait complete. 2023-07-06T21:39:30.399 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:39:30.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:30.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.605+0000 7ff89efff700 1 Processor -- start 2023-07-06T21:39:30.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.605+0000 7ff89efff700 1 -- start start 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89efff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8980b8530 0x7ff8980cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89efff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff8980d3580 0x7ff8980d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89efff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8980d0990 con 0x7ff8980cf3b0 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89efff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8980d3af0 con 0x7ff8980b8530 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89efff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8980cd9d0 con 0x7ff8980d3580 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89cd9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8980b8530 0x7ff8980cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89cd9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8980b8530 0x7ff8980cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38392/0 (socket says 172.21.15.84:38392) 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2681636456 learned_addr learned my addr 172.21.15.84:0/2681636456 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff897fff700 1 --2- 172.21.15.84:0/2681636456 >> v2:172.21.15.107:3300/0 conn(0x7ff8980d3580 0x7ff8980d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2681636456 >> v2:172.21.15.107:3300/0 conn(0x7ff8980d3580 msgr2=0x7ff8980d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89cd9b700 1 --2- 172.21.15.84:0/2681636456 >> v2:172.21.15.107:3300/0 conn(0x7ff8980d3580 0x7ff8980d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.606+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2681636456 >> v1:172.21.15.107:6789/0 conn(0x7ff8980cf3b0 legacy=0x7ff89815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2681636456 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980cd410 con 0x7ff8980b8530 2023-07-06T21:39:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff89cd9b700 1 --2- 172.21.15.84:0/2681636456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980cee70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff8980d07a0 tx=0x7ff88c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e9c925ab84082c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff8977fe700 1 -- 172.21.15.84:0/2681636456 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c005e30 con 0x7ff8980b8530 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff8977fe700 1 -- 172.21.15.84:0/2681636456 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c004030 con 0x7ff8980b8530 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff8977fe700 1 -- 172.21.15.84:0/2681636456 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0043e0 con 0x7ff8980b8530 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff89efff700 1 -- 172.21.15.84:0/2681636456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 msgr2=0x7ff8980cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.607+0000 7ff89efff700 1 --2- 172.21.15.84:0/2681636456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980cee70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff8980d07a0 tx=0x7ff88c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.608+0000 7ff89efff700 1 -- 172.21.15.84:0/2681636456 shutdown_connections 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.608+0000 7ff89efff700 1 --2- 172.21.15.84:0/2681636456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980cee70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.608+0000 7ff89efff700 1 --2- 172.21.15.84:0/2681636456 >> v2:172.21.15.107:3300/0 conn(0x7ff8980d3580 0x7ff8980d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.608+0000 7ff89efff700 1 -- 172.21.15.84:0/2681636456 >> 172.21.15.84:0/2681636456 conn(0x7ff8980715c0 msgr2=0x7ff898071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.608+0000 7ff89efff700 1 -- 172.21.15.84:0/2681636456 shutdown_connections 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.608+0000 7ff89efff700 1 -- 172.21.15.84:0/2681636456 wait complete. 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.609+0000 7ff89efff700 1 Processor -- start 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.610+0000 7ff89efff700 1 -- start start 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.610+0000 7ff89efff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.610+0000 7ff89efff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8980cf3b0 0x7ff8981e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.610+0000 7ff89efff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980d3580 0x7ff898154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89efff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8981e6210 con 0x7ff8980b8530 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89efff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980c7350 con 0x7ff8980d3580 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89efff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980c75a0 con 0x7ff8980cf3b0 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38404/0 (socket says 172.21.15.84:38404) 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2176846701 learned_addr learned my addr 172.21.15.84:0/2176846701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff897fff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8980cf3b0 0x7ff8981e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8980cf3b0 msgr2=0x7ff8981e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8980cf3b0 0x7ff8981e5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980d3580 msgr2=0x7ff898154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89d59c700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980d3580 0x7ff898154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980d3580 0x7ff898154620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 -- 172.21.15.84:0/2176846701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c009500 con 0x7ff8980b8530 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89d59c700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980d3580 0x7ff898154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:30.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.611+0000 7ff89cd9b700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980c9260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009c10 tx=0x7ff88c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:30.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.612+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c003dd0 con 0x7ff8980b8530 2023-07-06T21:39:30.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.612+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c011420 con 0x7ff8980b8530 2023-07-06T21:39:30.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.612+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c0117b0 con 0x7ff8980b8530 2023-07-06T21:39:30.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.612+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8980c7880 con 0x7ff8980b8530 2023-07-06T21:39:30.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.612+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8980c7e40 con 0x7ff8980b8530 2023-07-06T21:39:30.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.613+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89804e240 con 0x7ff8980b8530 2023-07-06T21:39:30.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.614+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff88c011910 con 0x7ff8980b8530 2023-07-06T21:39:30.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.615+0000 7ff8957fa700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff87c06ace0 0x7ff87c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:30.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.615+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff88c08a6e0 con 0x7ff8980b8530 2023-07-06T21:39:30.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.615+0000 7ff897fff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff87c06ace0 0x7ff87c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:30.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.615+0000 7ff897fff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff87c06ace0 0x7ff87c06d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8981e75c0 tx=0x7ff884008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:30.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.619+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff88c055820 con 0x7ff8980b8530 2023-07-06T21:39:30.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.818+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8980d2540 con 0x7ff8980b8530 2023-07-06T21:39:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.819+0000 7ff8957fa700 1 -- 172.21.15.84:0/2176846701 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7ff88c058e40 con 0x7ff8980b8530 2023-07-06T21:39:30.823 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:30.823 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff87c06ace0 msgr2=0x7ff87c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff87c06ace0 0x7ff87c06d190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8981e75c0 tx=0x7ff884008040 comp rx=0 tx=0).stop 2023-07-06T21:39:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 msgr2=0x7ff8980c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980c9260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff88c009c10 tx=0x7ff88c004750 comp rx=0 tx=0).stop 2023-07-06T21:39:30.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 shutdown_connections 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff87c06ace0 0x7ff87c06d190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8980d3580 0x7ff898154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8980b8530 0x7ff8980c9260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 --2- 172.21.15.84:0/2176846701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8980cf3b0 0x7ff8981e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 >> 172.21.15.84:0/2176846701 conn(0x7ff8980715c0 msgr2=0x7ff89815f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 shutdown_connections 2023-07-06T21:39:30.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:30.822+0000 7ff89efff700 1 -- 172.21.15.84:0/2176846701 wait complete. 2023-07-06T21:39:30.844 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T21:39:30.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:39:31.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.036+0000 7f49d01ff700 1 Processor -- start 2023-07-06T21:39:31.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.036+0000 7f49d01ff700 1 -- start start 2023-07-06T21:39:31.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49d01ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f49c80b83c0 0x7f49c80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49d01ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f49c80cfc60 0x7f49c80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49d01ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c80d0a80 con 0x7f49c80d3710 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49d01ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c80b8790 con 0x7f49c80cfc60 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49d01ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c80b4e70 con 0x7f49c80b83c0 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49cd79a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f49c80d3710 0x7f49c80cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52084/0 (socket says 172.21.15.84:52084) 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49cd79a700 1 -- 172.21.15.84:0/1070939980 learned_addr learned my addr 172.21.15.84:0/1070939980 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49ce79c700 1 --2- 172.21.15.84:0/1070939980 >> v2:172.21.15.84:3300/0 conn(0x7f49c80cfc60 0x7f49c80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.037+0000 7f49cdf9b700 1 --2- 172.21.15.84:0/1070939980 >> v2:172.21.15.107:3300/0 conn(0x7f49c80b83c0 0x7f49c80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ccf99700 1 -- 172.21.15.84:0/1070939980 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681560437 0 0) 0x7f49c80d0a80 con 0x7f49c80d3710 2023-07-06T21:39:31.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ce79c700 1 -- 172.21.15.84:0/1070939980 >> v2:172.21.15.107:3300/0 conn(0x7f49c80b83c0 msgr2=0x7f49c80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ce79c700 1 --2- 172.21.15.84:0/1070939980 >> v2:172.21.15.107:3300/0 conn(0x7f49c80b83c0 0x7f49c80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ce79c700 1 -- 172.21.15.84:0/1070939980 >> v1:172.21.15.84:6789/0 conn(0x7f49c80d3710 legacy=0x7f49c80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ce79c700 1 -- 172.21.15.84:0/1070939980 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c80b5150 con 0x7f49c80cfc60 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ce79c700 1 --2- 172.21.15.84:0/1070939980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80cfc60 0x7f49c80d0f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49c80cf520 tx=0x7f49c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cda47dee9e138e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ccf99700 1 -- 172.21.15.84:0/1070939980 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000e070 con 0x7f49c80cfc60 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ccf99700 1 -- 172.21.15.84:0/1070939980 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c000bbd0 con 0x7f49c80cfc60 2023-07-06T21:39:31.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.038+0000 7f49ccf99700 1 -- 172.21.15.84:0/1070939980 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0012430 con 0x7f49c80cfc60 2023-07-06T21:39:31.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 -- 172.21.15.84:0/1070939980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80cfc60 msgr2=0x7f49c80d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 --2- 172.21.15.84:0/1070939980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80cfc60 0x7f49c80d0f10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49c80cf520 tx=0x7f49c000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:31.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 -- 172.21.15.84:0/1070939980 shutdown_connections 2023-07-06T21:39:31.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 --2- 172.21.15.84:0/1070939980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80cfc60 0x7f49c80d0f10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 --2- 172.21.15.84:0/1070939980 >> v2:172.21.15.107:3300/0 conn(0x7f49c80b83c0 0x7f49c80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 -- 172.21.15.84:0/1070939980 >> 172.21.15.84:0/1070939980 conn(0x7f49c806b350 msgr2=0x7f49c806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 -- 172.21.15.84:0/1070939980 shutdown_connections 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.039+0000 7f49d01ff700 1 -- 172.21.15.84:0/1070939980 wait complete. 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.041+0000 7f49d01ff700 1 Processor -- start 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.041+0000 7f49d01ff700 1 -- start start 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49d01ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 0x7f49c80c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49d01ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c80cfc60 0x7f49c80b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49d01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c80d3710 0x7f49c80b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49d01ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c8156290 con 0x7f49c80b83c0 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49d01ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c8156530 con 0x7f49c80cfc60 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49d01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c8156810 con 0x7f49c80d3710 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 0x7f49c80c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 0x7f49c80c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38426/0 (socket says 172.21.15.84:38426) 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 -- 172.21.15.84:0/2374709997 learned_addr learned my addr 172.21.15.84:0/2374709997 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:31.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cd79a700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c80cfc60 0x7f49c80b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49ce79c700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c80d3710 0x7f49c80b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 -- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c80d3710 msgr2=0x7f49c80b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c80d3710 0x7f49c80b74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 -- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c80cfc60 msgr2=0x7f49c80b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c80cfc60 0x7f49c80b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 -- 172.21.15.84:0/2374709997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c0009a70 con 0x7f49c80b83c0 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.042+0000 7f49cdf9b700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 0x7f49c80c7380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49c400d800 tx=0x7f49c400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.043+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c40125f0 con 0x7f49c80b83c0 2023-07-06T21:39:31.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.043+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c4012750 con 0x7f49c80b83c0 2023-07-06T21:39:31.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.043+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4012b00 con 0x7f49c80b83c0 2023-07-06T21:39:31.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.043+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c8156af0 con 0x7f49c80b83c0 2023-07-06T21:39:31.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.043+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c80c78c0 con 0x7f49c80b83c0 2023-07-06T21:39:31.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.044+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c80c7a50 con 0x7f49c80b83c0 2023-07-06T21:39:31.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.045+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f49c4012c60 con 0x7f49c80b83c0 2023-07-06T21:39:31.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.046+0000 7f49be7fc700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.046+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f49c4089870 con 0x7f49c80b83c0 2023-07-06T21:39:31.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.046+0000 7f49cd79a700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.046+0000 7f49cd79a700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f49ac06ace0 0x7f49ac06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f49c8155c20 tx=0x7f49b8007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.050+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f49c4054910 con 0x7f49c80b83c0 2023-07-06T21:39:31.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.240+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f49c80d0710 con 0x7f49ac06ace0 2023-07-06T21:39:31.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.250+0000 7f49be7fc700 1 -- 172.21.15.84:0/2374709997 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286666 (secure 0 0 0) 0x7f49b80012b0 con 0x7f49ac06ace0 2023-07-06T21:39:31.257 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:31.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f49ac06ace0 msgr2=0x7f49ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f49ac06ace0 0x7f49ac06d190 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f49c8155c20 tx=0x7f49b8007be0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 msgr2=0x7f49c80c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 0x7f49c80c7380 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f49c400d800 tx=0x7f49c400dbc0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 shutdown_connections 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f49ac06ace0 0x7f49ac06d190 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c80d3710 0x7f49c80b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f49c80b83c0 0x7f49c80c7380 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 --2- 172.21.15.84:0/2374709997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c80cfc60 0x7f49c80b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 >> 172.21.15.84:0/2374709997 conn(0x7f49c806b350 msgr2=0x7f49c80b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 shutdown_connections 2023-07-06T21:39:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.255+0000 7f49d01ff700 1 -- 172.21.15.84:0/2374709997 wait complete. 2023-07-06T21:39:31.260 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:39:31.276 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-07-06T21:39:30.122797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218628,"kb_used_data":3012,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756092,"statfs":{"total":773094113280,"available":772870238208,"internally_reserved":0,"allocated":3084288,"data_stored":1848009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.198367"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459444+0000","last_change":"2023-07-06T21:39:26.450219+0000","last_active":"2023-07-06T21:39:26.459444+0000","last_peered":"2023-07-06T21:39:26.459444+0000","last_clean":"2023-07-06T21:39:26.459444+0000","last_became_active":"2023-07-06T21:39:26.449158+0000","last_became_peered":"2023-07-06T21:39:26.449158+0000","last_unstale":"2023-07-06T21:39:26.459444+0000","last_undegraded":"2023-07-06T21:39:26.459444+0000","last_fullsized":"2023-07-06T21:39:26.459444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464823+0000","last_change":"2023-07-06T21:39:26.456425+0000","last_active":"2023-07-06T21:39:26.464823+0000","last_peered":"2023-07-06T21:39:26.464823+0000","last_clean":"2023-07-06T21:39:26.464823+0000","last_became_active":"2023-07-06T21:39:26.455881+0000","last_became_peered":"2023-07-06T21:39:26.455881+0000","last_unstale":"2023-07-06T21:39:26.464823+0000","last_undegraded":"2023-07-06T21:39:26.464823+0000","last_fullsized":"2023-07-06T21:39:26.464823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:57.797206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457614+0000","last_change":"2023-07-06T21:39:26.448957+0000","last_active":"2023-07-06T21:39:26.457614+0000","last_peered":"2023-07-06T21:39:26.457614+0000","last_clean":"2023-07-06T21:39:26.457614+0000","last_became_active":"2023-07-06T21:39:26.448647+0000","last_became_peered":"2023-07-06T21:39:26.448647+0000","last_unstale":"2023-07-06T21:39:26.457614+0000","last_undegraded":"2023-07-06T21:39:26.457614+0000","last_fullsized":"2023-07-06T21:39:26.457614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:16.841715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470422+0000","last_change":"2023-07-06T21:39:26.468418+0000","last_active":"2023-07-06T21:39:26.470422+0000","last_peered":"2023-07-06T21:39:26.470422+0000","last_clean":"2023-07-06T21:39:26.470422+0000","last_became_active":"2023-07-06T21:39:26.464060+0000","last_became_peered":"2023-07-06T21:39:26.464060+0000","last_unstale":"2023-07-06T21:39:26.470422+0000","last_undegraded":"2023-07-06T21:39:26.470422+0000","last_fullsized":"2023-07-06T21:39:26.470422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465353+0000","last_change":"2023-07-06T21:39:26.461152+0000","last_active":"2023-07-06T21:39:26.465353+0000","last_peered":"2023-07-06T21:39:26.465353+0000","last_clean":"2023-07-06T21:39:26.465353+0000","last_became_active":"2023-07-06T21:39:26.460854+0000","last_became_peered":"2023-07-06T21:39:26.460854+0000","last_unstale":"2023-07-06T21:39:26.465353+0000","last_undegraded":"2023-07-06T21:39:26.465353+0000","last_fullsized":"2023-07-06T21:39:26.465353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457397+0000","last_change":"2023-07-06T21:39:26.449460+0000","last_active":"2023-07-06T21:39:26.457397+0000","last_peered":"2023-07-06T21:39:26.457397+0000","last_clean":"2023-07-06T21:39:26.457397+0000","last_became_active":"2023-07-06T21:39:26.449145+0000","last_became_peered":"2023-07-06T21:39:26.449145+0000","last_unstale":"2023-07-06T21:39:26.457397+0000","last_undegraded":"2023-07-06T21:39:26.457397+0000","last_fullsized":"2023-07-06T21:39:26.457397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469794+0000","last_change":"2023-07-06T21:39:26.467462+0000","last_active":"2023-07-06T21:39:26.469794+0000","last_peered":"2023-07-06T21:39:26.469794+0000","last_clean":"2023-07-06T21:39:26.469794+0000","last_became_active":"2023-07-06T21:39:26.466827+0000","last_became_peered":"2023-07-06T21:39:26.466827+0000","last_unstale":"2023-07-06T21:39:26.469794+0000","last_undegraded":"2023-07-06T21:39:26.469794+0000","last_fullsized":"2023-07-06T21:39:26.469794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:11.539304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.467206+0000","last_change":"2023-07-06T21:39:26.459017+0000","last_active":"2023-07-06T21:39:26.467206+0000","last_peered":"2023-07-06T21:39:26.467206+0000","last_clean":"2023-07-06T21:39:26.467206+0000","last_became_active":"2023-07-06T21:39:26.458749+0000","last_became_peered":"2023-07-06T21:39:26.458749+0000","last_unstale":"2023-07-06T21:39:26.467206+0000","last_undegraded":"2023-07-06T21:39:26.467206+0000","last_fullsized":"2023-07-06T21:39:26.467206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461280+0000","last_change":"2023-07-06T21:39:26.453377+0000","last_active":"2023-07-06T21:39:26.461280+0000","last_peered":"2023-07-06T21:39:26.461280+0000","last_clean":"2023-07-06T21:39:26.461280+0000","last_became_active":"2023-07-06T21:39:26.452864+0000","last_became_peered":"2023-07-06T21:39:26.452864+0000","last_unstale":"2023-07-06T21:39:26.461280+0000","last_undegraded":"2023-07-06T21:39:26.461280+0000","last_fullsized":"2023-07-06T21:39:26.461280+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464800+0000","last_change":"2023-07-06T21:39:26.456191+0000","last_active":"2023-07-06T21:39:26.464800+0000","last_peered":"2023-07-06T21:39:26.464800+0000","last_clean":"2023-07-06T21:39:26.464800+0000","last_became_active":"2023-07-06T21:39:26.455459+0000","last_became_peered":"2023-07-06T21:39:26.455459+0000","last_unstale":"2023-07-06T21:39:26.464800+0000","last_undegraded":"2023-07-06T21:39:26.464800+0000","last_fullsized":"2023-07-06T21:39:26.464800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:48.843759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457676+0000","last_change":"2023-07-06T21:39:26.450168+0000","last_active":"2023-07-06T21:39:26.457676+0000","last_peered":"2023-07-06T21:39:26.457676+0000","last_clean":"2023-07-06T21:39:26.457676+0000","last_became_active":"2023-07-06T21:39:26.449224+0000","last_became_peered":"2023-07-06T21:39:26.449224+0000","last_unstale":"2023-07-06T21:39:26.457676+0000","last_undegraded":"2023-07-06T21:39:26.457676+0000","last_fullsized":"2023-07-06T21:39:26.457676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461192+0000","last_change":"2023-07-06T21:39:26.454525+0000","last_active":"2023-07-06T21:39:26.461192+0000","last_peered":"2023-07-06T21:39:26.461192+0000","last_clean":"2023-07-06T21:39:26.461192+0000","last_became_active":"2023-07-06T21:39:26.453872+0000","last_became_peered":"2023-07-06T21:39:26.453872+0000","last_unstale":"2023-07-06T21:39:26.461192+0000","last_undegraded":"2023-07-06T21:39:26.461192+0000","last_fullsized":"2023-07-06T21:39:26.461192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459328+0000","last_change":"2023-07-06T21:39:26.449670+0000","last_active":"2023-07-06T21:39:26.459328+0000","last_peered":"2023-07-06T21:39:26.459328+0000","last_clean":"2023-07-06T21:39:26.459328+0000","last_became_active":"2023-07-06T21:39:26.449116+0000","last_became_peered":"2023-07-06T21:39:26.449116+0000","last_unstale":"2023-07-06T21:39:26.459328+0000","last_undegraded":"2023-07-06T21:39:26.459328+0000","last_fullsized":"2023-07-06T21:39:26.459328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:33.107986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.460744+0000","last_change":"2023-07-06T21:39:26.450117+0000","last_active":"2023-07-06T21:39:26.460744+0000","last_peered":"2023-07-06T21:39:26.460744+0000","last_clean":"2023-07-06T21:39:26.460744+0000","last_became_active":"2023-07-06T21:39:26.449800+0000","last_became_peered":"2023-07-06T21:39:26.449800+0000","last_unstale":"2023-07-06T21:39:26.460744+0000","last_undegraded":"2023-07-06T21:39:26.460744+0000","last_fullsized":"2023-07-06T21:39:26.460744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469812+0000","last_change":"2023-07-06T21:39:26.467633+0000","last_active":"2023-07-06T21:39:26.469812+0000","last_peered":"2023-07-06T21:39:26.469812+0000","last_clean":"2023-07-06T21:39:26.469812+0000","last_became_active":"2023-07-06T21:39:26.467194+0000","last_became_peered":"2023-07-06T21:39:26.467194+0000","last_unstale":"2023-07-06T21:39:26.469812+0000","last_undegraded":"2023-07-06T21:39:26.469812+0000","last_fullsized":"2023-07-06T21:39:26.469812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:48.843759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459075+0000","last_change":"2023-07-06T21:39:26.448432+0000","last_active":"2023-07-06T21:39:26.459075+0000","last_peered":"2023-07-06T21:39:26.459075+0000","last_clean":"2023-07-06T21:39:26.459075+0000","last_became_active":"2023-07-06T21:39:26.448091+0000","last_became_peered":"2023-07-06T21:39:26.448091+0000","last_unstale":"2023-07-06T21:39:26.459075+0000","last_undegraded":"2023-07-06T21:39:26.459075+0000","last_fullsized":"2023-07-06T21:39:26.459075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:28.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461260+0000","last_change":"2023-07-06T21:39:26.453156+0000","last_active":"2023-07-06T21:39:26.461260+0000","last_peered":"2023-07-06T21:39:26.461260+0000","last_clean":"2023-07-06T21:39:26.461260+0000","last_became_active":"2023-07-06T21:39:26.452499+0000","last_became_peered":"2023-07-06T21:39:26.452499+0000","last_unstale":"2023-07-06T21:39:26.461260+0000","last_undegraded":"2023-07-06T21:39:26.461260+0000","last_fullsized":"2023-07-06T21:39:26.461260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:33.107986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.454139+0000","last_change":"2023-07-06T21:39:26.442470+0000","last_active":"2023-07-06T21:39:26.454139+0000","last_peered":"2023-07-06T21:39:26.454139+0000","last_clean":"2023-07-06T21:39:26.454139+0000","last_became_active":"2023-07-06T21:39:26.442162+0000","last_became_peered":"2023-07-06T21:39:26.442162+0000","last_unstale":"2023-07-06T21:39:26.454139+0000","last_undegraded":"2023-07-06T21:39:26.454139+0000","last_fullsized":"2023-07-06T21:39:26.454139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473217+0000","last_change":"2023-07-06T21:39:26.469782+0000","last_active":"2023-07-06T21:39:26.473217+0000","last_peered":"2023-07-06T21:39:26.473217+0000","last_clean":"2023-07-06T21:39:26.473217+0000","last_became_active":"2023-07-06T21:39:26.469492+0000","last_became_peered":"2023-07-06T21:39:26.469492+0000","last_unstale":"2023-07-06T21:39:26.473217+0000","last_undegraded":"2023-07-06T21:39:26.473217+0000","last_fullsized":"2023-07-06T21:39:26.473217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.472585+0000","last_change":"2023-07-06T21:39:26.465478+0000","last_active":"2023-07-06T21:39:26.472585+0000","last_peered":"2023-07-06T21:39:26.472585+0000","last_clean":"2023-07-06T21:39:26.472585+0000","last_became_active":"2023-07-06T21:39:26.465171+0000","last_became_peered":"2023-07-06T21:39:26.465171+0000","last_unstale":"2023-07-06T21:39:26.472585+0000","last_undegraded":"2023-07-06T21:39:26.472585+0000","last_fullsized":"2023-07-06T21:39:26.472585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:05.032248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470274+0000","last_change":"2023-07-06T21:39:26.465956+0000","last_active":"2023-07-06T21:39:26.470274+0000","last_peered":"2023-07-06T21:39:26.470274+0000","last_clean":"2023-07-06T21:39:26.470274+0000","last_became_active":"2023-07-06T21:39:26.465672+0000","last_became_peered":"2023-07-06T21:39:26.465672+0000","last_unstale":"2023-07-06T21:39:26.470274+0000","last_undegraded":"2023-07-06T21:39:26.470274+0000","last_fullsized":"2023-07-06T21:39:26.470274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:28.800563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461210+0000","last_change":"2023-07-06T21:39:26.453279+0000","last_active":"2023-07-06T21:39:26.461210+0000","last_peered":"2023-07-06T21:39:26.461210+0000","last_clean":"2023-07-06T21:39:26.461210+0000","last_became_active":"2023-07-06T21:39:26.452832+0000","last_became_peered":"2023-07-06T21:39:26.452832+0000","last_unstale":"2023-07-06T21:39:26.461210+0000","last_undegraded":"2023-07-06T21:39:26.461210+0000","last_fullsized":"2023-07-06T21:39:26.461210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:58:45.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462541+0000","last_change":"2023-07-06T21:39:26.450178+0000","last_active":"2023-07-06T21:39:26.462541+0000","last_peered":"2023-07-06T21:39:26.462541+0000","last_clean":"2023-07-06T21:39:26.462541+0000","last_became_active":"2023-07-06T21:39:26.449355+0000","last_became_peered":"2023-07-06T21:39:26.449355+0000","last_unstale":"2023-07-06T21:39:26.462541+0000","last_undegraded":"2023-07-06T21:39:26.462541+0000","last_fullsized":"2023-07-06T21:39:26.462541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.467846+0000","last_change":"2023-07-06T21:39:26.465115+0000","last_active":"2023-07-06T21:39:26.467846+0000","last_peered":"2023-07-06T21:39:26.467846+0000","last_clean":"2023-07-06T21:39:26.467846+0000","last_became_active":"2023-07-06T21:39:26.464450+0000","last_became_peered":"2023-07-06T21:39:26.464450+0000","last_unstale":"2023-07-06T21:39:26.467846+0000","last_undegraded":"2023-07-06T21:39:26.467846+0000","last_fullsized":"2023-07-06T21:39:26.467846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466037+0000","last_change":"2023-07-06T21:39:26.464834+0000","last_active":"2023-07-06T21:39:26.466037+0000","last_peered":"2023-07-06T21:39:26.466037+0000","last_clean":"2023-07-06T21:39:26.466037+0000","last_became_active":"2023-07-06T21:39:26.464519+0000","last_became_peered":"2023-07-06T21:39:26.464519+0000","last_unstale":"2023-07-06T21:39:26.466037+0000","last_undegraded":"2023-07-06T21:39:26.466037+0000","last_fullsized":"2023-07-06T21:39:26.466037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:08.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461455+0000","last_change":"2023-07-06T21:39:26.451689+0000","last_active":"2023-07-06T21:39:26.461455+0000","last_peered":"2023-07-06T21:39:26.461455+0000","last_clean":"2023-07-06T21:39:26.461455+0000","last_became_active":"2023-07-06T21:39:26.451397+0000","last_became_peered":"2023-07-06T21:39:26.451397+0000","last_unstale":"2023-07-06T21:39:26.461455+0000","last_undegraded":"2023-07-06T21:39:26.461455+0000","last_fullsized":"2023-07-06T21:39:26.461455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468459+0000","last_change":"2023-07-06T21:39:26.466401+0000","last_active":"2023-07-06T21:39:26.468459+0000","last_peered":"2023-07-06T21:39:26.468459+0000","last_clean":"2023-07-06T21:39:26.468459+0000","last_became_active":"2023-07-06T21:39:26.465938+0000","last_became_peered":"2023-07-06T21:39:26.465938+0000","last_unstale":"2023-07-06T21:39:26.468459+0000","last_undegraded":"2023-07-06T21:39:26.468459+0000","last_fullsized":"2023-07-06T21:39:26.468459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457230+0000","last_change":"2023-07-06T21:39:25.954109+0000","last_active":"2023-07-06T21:39:26.457230+0000","last_peered":"2023-07-06T21:39:26.457230+0000","last_clean":"2023-07-06T21:39:26.457230+0000","last_became_active":"2023-07-06T21:39:25.953788+0000","last_became_peered":"2023-07-06T21:39:25.953788+0000","last_unstale":"2023-07-06T21:39:26.457230+0000","last_undegraded":"2023-07-06T21:39:26.457230+0000","last_fullsized":"2023-07-06T21:39:26.457230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462717+0000","last_change":"2023-07-06T21:39:26.455786+0000","last_active":"2023-07-06T21:39:26.462717+0000","last_peered":"2023-07-06T21:39:26.462717+0000","last_clean":"2023-07-06T21:39:26.462717+0000","last_became_active":"2023-07-06T21:39:26.455415+0000","last_became_peered":"2023-07-06T21:39:26.455415+0000","last_unstale":"2023-07-06T21:39:26.462717+0000","last_undegraded":"2023-07-06T21:39:26.462717+0000","last_fullsized":"2023-07-06T21:39:26.462717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:18.739171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.446240+0000","last_change":"2023-07-06T21:39:25.952238+0000","last_active":"2023-07-06T21:39:26.446240+0000","last_peered":"2023-07-06T21:39:26.446240+0000","last_clean":"2023-07-06T21:39:26.446240+0000","last_became_active":"2023-07-06T21:39:25.951890+0000","last_became_peered":"2023-07-06T21:39:25.951890+0000","last_unstale":"2023-07-06T21:39:26.446240+0000","last_undegraded":"2023-07-06T21:39:26.446240+0000","last_fullsized":"2023-07-06T21:39:26.446240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465983+0000","last_change":"2023-07-06T21:39:26.463846+0000","last_active":"2023-07-06T21:39:26.465983+0000","last_peered":"2023-07-06T21:39:26.465983+0000","last_clean":"2023-07-06T21:39:26.465983+0000","last_became_active":"2023-07-06T21:39:26.463530+0000","last_became_peered":"2023-07-06T21:39:26.463530+0000","last_unstale":"2023-07-06T21:39:26.465983+0000","last_undegraded":"2023-07-06T21:39:26.465983+0000","last_fullsized":"2023-07-06T21:39:26.465983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447565+0000","last_change":"2023-07-06T21:39:25.958386+0000","last_active":"2023-07-06T21:39:26.447565+0000","last_peered":"2023-07-06T21:39:26.447565+0000","last_clean":"2023-07-06T21:39:26.447565+0000","last_became_active":"2023-07-06T21:39:25.958049+0000","last_became_peered":"2023-07-06T21:39:25.958049+0000","last_unstale":"2023-07-06T21:39:26.447565+0000","last_undegraded":"2023-07-06T21:39:26.447565+0000","last_fullsized":"2023-07-06T21:39:26.447565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.463897+0000","last_change":"2023-07-06T21:39:26.461047+0000","last_active":"2023-07-06T21:39:26.463897+0000","last_peered":"2023-07-06T21:39:26.463897+0000","last_clean":"2023-07-06T21:39:26.463897+0000","last_became_active":"2023-07-06T21:39:26.460608+0000","last_became_peered":"2023-07-06T21:39:26.460608+0000","last_unstale":"2023-07-06T21:39:26.463897+0000","last_undegraded":"2023-07-06T21:39:26.463897+0000","last_fullsized":"2023-07-06T21:39:26.463897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442814+0000","last_change":"2023-07-06T21:39:25.953273+0000","last_active":"2023-07-06T21:39:26.442814+0000","last_peered":"2023-07-06T21:39:26.442814+0000","last_clean":"2023-07-06T21:39:26.442814+0000","last_became_active":"2023-07-06T21:39:25.947023+0000","last_became_peered":"2023-07-06T21:39:25.947023+0000","last_unstale":"2023-07-06T21:39:26.442814+0000","last_undegraded":"2023-07-06T21:39:26.442814+0000","last_fullsized":"2023-07-06T21:39:26.442814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:43.182084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462635+0000","last_change":"2023-07-06T21:39:26.453860+0000","last_active":"2023-07-06T21:39:26.462635+0000","last_peered":"2023-07-06T21:39:26.462635+0000","last_clean":"2023-07-06T21:39:26.462635+0000","last_became_active":"2023-07-06T21:39:26.453597+0000","last_became_peered":"2023-07-06T21:39:26.453597+0000","last_unstale":"2023-07-06T21:39:26.462635+0000","last_undegraded":"2023-07-06T21:39:26.462635+0000","last_fullsized":"2023-07-06T21:39:26.462635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:15.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.443836+0000","last_change":"2023-07-06T21:39:25.949777+0000","last_active":"2023-07-06T21:39:26.443836+0000","last_peered":"2023-07-06T21:39:26.443836+0000","last_clean":"2023-07-06T21:39:26.443836+0000","last_became_active":"2023-07-06T21:39:25.949379+0000","last_became_peered":"2023-07-06T21:39:25.949379+0000","last_unstale":"2023-07-06T21:39:26.443836+0000","last_undegraded":"2023-07-06T21:39:26.443836+0000","last_fullsized":"2023-07-06T21:39:26.443836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:10.672168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465950+0000","last_change":"2023-07-06T21:39:26.463815+0000","last_active":"2023-07-06T21:39:26.465950+0000","last_peered":"2023-07-06T21:39:26.465950+0000","last_clean":"2023-07-06T21:39:26.465950+0000","last_became_active":"2023-07-06T21:39:26.463557+0000","last_became_peered":"2023-07-06T21:39:26.463557+0000","last_unstale":"2023-07-06T21:39:26.465950+0000","last_undegraded":"2023-07-06T21:39:26.465950+0000","last_fullsized":"2023-07-06T21:39:26.465950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466327+0000","last_change":"2023-07-06T21:39:25.963442+0000","last_active":"2023-07-06T21:39:26.466327+0000","last_peered":"2023-07-06T21:39:26.466327+0000","last_clean":"2023-07-06T21:39:26.466327+0000","last_became_active":"2023-07-06T21:39:25.963058+0000","last_became_peered":"2023-07-06T21:39:25.963058+0000","last_unstale":"2023-07-06T21:39:26.466327+0000","last_undegraded":"2023-07-06T21:39:26.466327+0000","last_fullsized":"2023-07-06T21:39:26.466327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:10.672168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465705+0000","last_change":"2023-07-06T21:39:26.464284+0000","last_active":"2023-07-06T21:39:26.465705+0000","last_peered":"2023-07-06T21:39:26.465705+0000","last_clean":"2023-07-06T21:39:26.465705+0000","last_became_active":"2023-07-06T21:39:26.463874+0000","last_became_peered":"2023-07-06T21:39:26.463874+0000","last_unstale":"2023-07-06T21:39:26.465705+0000","last_undegraded":"2023-07-06T21:39:26.465705+0000","last_fullsized":"2023-07-06T21:39:26.465705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.474140+0000","last_change":"2023-07-06T21:39:25.950225+0000","last_active":"2023-07-06T21:39:26.474140+0000","last_peered":"2023-07-06T21:39:26.474140+0000","last_clean":"2023-07-06T21:39:26.474140+0000","last_became_active":"2023-07-06T21:39:25.949924+0000","last_became_peered":"2023-07-06T21:39:25.949924+0000","last_unstale":"2023-07-06T21:39:26.474140+0000","last_undegraded":"2023-07-06T21:39:26.474140+0000","last_fullsized":"2023-07-06T21:39:26.474140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:06.297911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465101+0000","last_change":"2023-07-06T21:39:26.463354+0000","last_active":"2023-07-06T21:39:26.465101+0000","last_peered":"2023-07-06T21:39:26.465101+0000","last_clean":"2023-07-06T21:39:26.465101+0000","last_became_active":"2023-07-06T21:39:26.463039+0000","last_became_peered":"2023-07-06T21:39:26.463039+0000","last_unstale":"2023-07-06T21:39:26.465101+0000","last_undegraded":"2023-07-06T21:39:26.465101+0000","last_fullsized":"2023-07-06T21:39:26.465101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:33.107986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469446+0000","last_change":"2023-07-06T21:39:25.951134+0000","last_active":"2023-07-06T21:39:26.469446+0000","last_peered":"2023-07-06T21:39:26.469446+0000","last_clean":"2023-07-06T21:39:26.469446+0000","last_became_active":"2023-07-06T21:39:25.950821+0000","last_became_peered":"2023-07-06T21:39:25.950821+0000","last_unstale":"2023-07-06T21:39:26.469446+0000","last_undegraded":"2023-07-06T21:39:26.469446+0000","last_fullsized":"2023-07-06T21:39:26.469446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462736+0000","last_change":"2023-07-06T21:39:26.455921+0000","last_active":"2023-07-06T21:39:26.462736+0000","last_peered":"2023-07-06T21:39:26.462736+0000","last_clean":"2023-07-06T21:39:26.462736+0000","last_became_active":"2023-07-06T21:39:26.455574+0000","last_became_peered":"2023-07-06T21:39:26.455574+0000","last_unstale":"2023-07-06T21:39:26.462736+0000","last_undegraded":"2023-07-06T21:39:26.462736+0000","last_fullsized":"2023-07-06T21:39:26.462736+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442838+0000","last_change":"2023-07-06T21:39:25.949803+0000","last_active":"2023-07-06T21:39:26.442838+0000","last_peered":"2023-07-06T21:39:26.442838+0000","last_clean":"2023-07-06T21:39:26.442838+0000","last_became_active":"2023-07-06T21:39:25.949490+0000","last_became_peered":"2023-07-06T21:39:25.949490+0000","last_unstale":"2023-07-06T21:39:26.442838+0000","last_undegraded":"2023-07-06T21:39:26.442838+0000","last_fullsized":"2023-07-06T21:39:26.442838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:38.093027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468440+0000","last_change":"2023-07-06T21:39:26.466204+0000","last_active":"2023-07-06T21:39:26.468440+0000","last_peered":"2023-07-06T21:39:26.468440+0000","last_clean":"2023-07-06T21:39:26.468440+0000","last_became_active":"2023-07-06T21:39:26.465576+0000","last_became_peered":"2023-07-06T21:39:26.465576+0000","last_unstale":"2023-07-06T21:39:26.468440+0000","last_undegraded":"2023-07-06T21:39:26.468440+0000","last_fullsized":"2023-07-06T21:39:26.468440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:16.841715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459462+0000","last_change":"2023-07-06T21:39:25.952231+0000","last_active":"2023-07-06T21:39:26.459462+0000","last_peered":"2023-07-06T21:39:26.459462+0000","last_clean":"2023-07-06T21:39:26.459462+0000","last_became_active":"2023-07-06T21:39:25.951945+0000","last_became_peered":"2023-07-06T21:39:25.951945+0000","last_unstale":"2023-07-06T21:39:26.459462+0000","last_undegraded":"2023-07-06T21:39:26.459462+0000","last_fullsized":"2023-07-06T21:39:26.459462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:55.145873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465391+0000","last_change":"2023-07-06T21:39:26.460570+0000","last_active":"2023-07-06T21:39:26.465391+0000","last_peered":"2023-07-06T21:39:26.465391+0000","last_clean":"2023-07-06T21:39:26.465391+0000","last_became_active":"2023-07-06T21:39:26.460310+0000","last_became_peered":"2023-07-06T21:39:26.460310+0000","last_unstale":"2023-07-06T21:39:26.465391+0000","last_undegraded":"2023-07-06T21:39:26.465391+0000","last_fullsized":"2023-07-06T21:39:26.465391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:28.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469415+0000","last_change":"2023-07-06T21:39:25.953216+0000","last_active":"2023-07-06T21:39:26.469415+0000","last_peered":"2023-07-06T21:39:26.469415+0000","last_clean":"2023-07-06T21:39:26.469415+0000","last_became_active":"2023-07-06T21:39:25.952808+0000","last_became_peered":"2023-07-06T21:39:25.952808+0000","last_unstale":"2023-07-06T21:39:26.469415+0000","last_undegraded":"2023-07-06T21:39:26.469415+0000","last_fullsized":"2023-07-06T21:39:26.469415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:14.829725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465918+0000","last_change":"2023-07-06T21:39:26.463251+0000","last_active":"2023-07-06T21:39:26.465918+0000","last_peered":"2023-07-06T21:39:26.465918+0000","last_clean":"2023-07-06T21:39:26.465918+0000","last_became_active":"2023-07-06T21:39:26.463003+0000","last_became_peered":"2023-07-06T21:39:26.463003+0000","last_unstale":"2023-07-06T21:39:26.465918+0000","last_undegraded":"2023-07-06T21:39:26.465918+0000","last_fullsized":"2023-07-06T21:39:26.465918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462670+0000","last_change":"2023-07-06T21:39:25.950786+0000","last_active":"2023-07-06T21:39:26.462670+0000","last_peered":"2023-07-06T21:39:26.462670+0000","last_clean":"2023-07-06T21:39:26.462670+0000","last_became_active":"2023-07-06T21:39:25.950367+0000","last_became_peered":"2023-07-06T21:39:25.950367+0000","last_unstale":"2023-07-06T21:39:26.462670+0000","last_undegraded":"2023-07-06T21:39:26.462670+0000","last_fullsized":"2023-07-06T21:39:26.462670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:56.789261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.467393+0000","last_change":"2023-07-06T21:39:26.465718+0000","last_active":"2023-07-06T21:39:26.467393+0000","last_peered":"2023-07-06T21:39:26.467393+0000","last_clean":"2023-07-06T21:39:26.467393+0000","last_became_active":"2023-07-06T21:39:26.465470+0000","last_became_peered":"2023-07-06T21:39:26.465470+0000","last_unstale":"2023-07-06T21:39:26.467393+0000","last_undegraded":"2023-07-06T21:39:26.467393+0000","last_fullsized":"2023-07-06T21:39:26.467393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462581+0000","last_change":"2023-07-06T21:39:25.950737+0000","last_active":"2023-07-06T21:39:26.462581+0000","last_peered":"2023-07-06T21:39:26.462581+0000","last_clean":"2023-07-06T21:39:26.462581+0000","last_became_active":"2023-07-06T21:39:25.950252+0000","last_became_peered":"2023-07-06T21:39:25.950252+0000","last_unstale":"2023-07-06T21:39:26.462581+0000","last_undegraded":"2023-07-06T21:39:26.462581+0000","last_fullsized":"2023-07-06T21:39:26.462581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:06.297911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457222+0000","last_change":"2023-07-06T21:39:26.449765+0000","last_active":"2023-07-06T21:39:26.457222+0000","last_peered":"2023-07-06T21:39:26.457222+0000","last_clean":"2023-07-06T21:39:26.457222+0000","last_became_active":"2023-07-06T21:39:26.449453+0000","last_became_peered":"2023-07-06T21:39:26.449453+0000","last_unstale":"2023-07-06T21:39:26.457222+0000","last_undegraded":"2023-07-06T21:39:26.457222+0000","last_fullsized":"2023-07-06T21:39:26.457222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468835+0000","last_change":"2023-07-06T21:39:25.951600+0000","last_active":"2023-07-06T21:39:26.468835+0000","last_peered":"2023-07-06T21:39:26.468835+0000","last_clean":"2023-07-06T21:39:26.468835+0000","last_became_active":"2023-07-06T21:39:25.951237+0000","last_became_peered":"2023-07-06T21:39:25.951237+0000","last_unstale":"2023-07-06T21:39:26.468835+0000","last_undegraded":"2023-07-06T21:39:26.468835+0000","last_fullsized":"2023-07-06T21:39:26.468835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:35.274536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470358+0000","last_change":"2023-07-06T21:39:26.466709+0000","last_active":"2023-07-06T21:39:26.470358+0000","last_peered":"2023-07-06T21:39:26.470358+0000","last_clean":"2023-07-06T21:39:26.470358+0000","last_became_active":"2023-07-06T21:39:26.463332+0000","last_became_peered":"2023-07-06T21:39:26.463332+0000","last_unstale":"2023-07-06T21:39:26.470358+0000","last_undegraded":"2023-07-06T21:39:26.470358+0000","last_fullsized":"2023-07-06T21:39:26.470358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442593+0000","last_change":"2023-07-06T21:39:25.953896+0000","last_active":"2023-07-06T21:39:26.442593+0000","last_peered":"2023-07-06T21:39:26.442593+0000","last_clean":"2023-07-06T21:39:26.442593+0000","last_became_active":"2023-07-06T21:39:25.953607+0000","last_became_peered":"2023-07-06T21:39:25.953607+0000","last_unstale":"2023-07-06T21:39:26.442593+0000","last_undegraded":"2023-07-06T21:39:26.442593+0000","last_fullsized":"2023-07-06T21:39:26.442593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461226+0000","last_change":"2023-07-06T21:39:26.453327+0000","last_active":"2023-07-06T21:39:26.461226+0000","last_peered":"2023-07-06T21:39:26.461226+0000","last_clean":"2023-07-06T21:39:26.461226+0000","last_became_active":"2023-07-06T21:39:26.453059+0000","last_became_peered":"2023-07-06T21:39:26.453059+0000","last_unstale":"2023-07-06T21:39:26.461226+0000","last_undegraded":"2023-07-06T21:39:26.461226+0000","last_fullsized":"2023-07-06T21:39:26.461226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:15.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466101+0000","last_change":"2023-07-06T21:39:25.955876+0000","last_active":"2023-07-06T21:39:26.466101+0000","last_peered":"2023-07-06T21:39:26.466101+0000","last_clean":"2023-07-06T21:39:26.466101+0000","last_became_active":"2023-07-06T21:39:25.955527+0000","last_became_peered":"2023-07-06T21:39:25.955527+0000","last_unstale":"2023-07-06T21:39:26.466101+0000","last_undegraded":"2023-07-06T21:39:26.466101+0000","last_fullsized":"2023-07-06T21:39:26.466101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:32.946775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459049+0000","last_change":"2023-07-06T21:39:26.452205+0000","last_active":"2023-07-06T21:39:26.459049+0000","last_peered":"2023-07-06T21:39:26.459049+0000","last_clean":"2023-07-06T21:39:26.459049+0000","last_became_active":"2023-07-06T21:39:26.451938+0000","last_became_peered":"2023-07-06T21:39:26.451938+0000","last_unstale":"2023-07-06T21:39:26.459049+0000","last_undegraded":"2023-07-06T21:39:26.459049+0000","last_fullsized":"2023-07-06T21:39:26.459049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447669+0000","last_change":"2023-07-06T21:39:25.954596+0000","last_active":"2023-07-06T21:39:26.447669+0000","last_peered":"2023-07-06T21:39:26.447669+0000","last_clean":"2023-07-06T21:39:26.447669+0000","last_became_active":"2023-07-06T21:39:25.954316+0000","last_became_peered":"2023-07-06T21:39:25.954316+0000","last_unstale":"2023-07-06T21:39:26.447669+0000","last_undegraded":"2023-07-06T21:39:26.447669+0000","last_fullsized":"2023-07-06T21:39:26.447669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:27.268839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457592+0000","last_change":"2023-07-06T21:39:26.446371+0000","last_active":"2023-07-06T21:39:26.457592+0000","last_peered":"2023-07-06T21:39:26.457592+0000","last_clean":"2023-07-06T21:39:26.457592+0000","last_became_active":"2023-07-06T21:39:26.445294+0000","last_became_peered":"2023-07-06T21:39:26.445294+0000","last_unstale":"2023-07-06T21:39:26.457592+0000","last_undegraded":"2023-07-06T21:39:26.457592+0000","last_fullsized":"2023-07-06T21:39:26.457592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.437948+0000","last_change":"2023-07-06T21:39:25.954319+0000","last_active":"2023-07-06T21:39:26.437948+0000","last_peered":"2023-07-06T21:39:26.437948+0000","last_clean":"2023-07-06T21:39:26.437948+0000","last_became_active":"2023-07-06T21:39:25.954089+0000","last_became_peered":"2023-07-06T21:39:25.954089+0000","last_unstale":"2023-07-06T21:39:26.437948+0000","last_undegraded":"2023-07-06T21:39:26.437948+0000","last_fullsized":"2023-07-06T21:39:26.437948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465097+0000","last_change":"2023-07-06T21:39:26.463246+0000","last_active":"2023-07-06T21:39:26.465097+0000","last_peered":"2023-07-06T21:39:26.465097+0000","last_clean":"2023-07-06T21:39:26.465097+0000","last_became_active":"2023-07-06T21:39:26.462881+0000","last_became_peered":"2023-07-06T21:39:26.462881+0000","last_unstale":"2023-07-06T21:39:26.465097+0000","last_undegraded":"2023-07-06T21:39:26.465097+0000","last_fullsized":"2023-07-06T21:39:26.465097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447412+0000","last_change":"2023-07-06T21:39:25.962410+0000","last_active":"2023-07-06T21:39:26.447412+0000","last_peered":"2023-07-06T21:39:26.447412+0000","last_clean":"2023-07-06T21:39:26.447412+0000","last_became_active":"2023-07-06T21:39:25.962150+0000","last_became_peered":"2023-07-06T21:39:25.962150+0000","last_unstale":"2023-07-06T21:39:26.447412+0000","last_undegraded":"2023-07-06T21:39:26.447412+0000","last_fullsized":"2023-07-06T21:39:26.447412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.456964+0000","last_change":"2023-07-06T21:39:26.449718+0000","last_active":"2023-07-06T21:39:26.456964+0000","last_peered":"2023-07-06T21:39:26.456964+0000","last_clean":"2023-07-06T21:39:26.456964+0000","last_became_active":"2023-07-06T21:39:26.449444+0000","last_became_peered":"2023-07-06T21:39:26.449444+0000","last_unstale":"2023-07-06T21:39:26.456964+0000","last_undegraded":"2023-07-06T21:39:26.456964+0000","last_fullsized":"2023-07-06T21:39:26.456964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:18.739171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447717+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:39:26.447717+0000","last_peered":"2023-07-06T21:39:26.447717+0000","last_clean":"2023-07-06T21:39:26.447717+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:39:26.447717+0000","last_undegraded":"2023-07-06T21:39:26.447717+0000","last_fullsized":"2023-07-06T21:39:26.447717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:42.417319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442354+0000","last_change":"2023-07-06T21:39:25.962785+0000","last_active":"2023-07-06T21:39:26.442354+0000","last_peered":"2023-07-06T21:39:26.442354+0000","last_clean":"2023-07-06T21:39:26.442354+0000","last_became_active":"2023-07-06T21:39:25.962456+0000","last_became_peered":"2023-07-06T21:39:25.962456+0000","last_unstale":"2023-07-06T21:39:26.442354+0000","last_undegraded":"2023-07-06T21:39:26.442354+0000","last_fullsized":"2023-07-06T21:39:26.442354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465217+0000","last_change":"2023-07-06T21:39:26.457017+0000","last_active":"2023-07-06T21:39:26.465217+0000","last_peered":"2023-07-06T21:39:26.465217+0000","last_clean":"2023-07-06T21:39:26.465217+0000","last_became_active":"2023-07-06T21:39:26.456767+0000","last_became_peered":"2023-07-06T21:39:26.456767+0000","last_unstale":"2023-07-06T21:39:26.465217+0000","last_undegraded":"2023-07-06T21:39:26.465217+0000","last_fullsized":"2023-07-06T21:39:26.465217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.441066+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:39:26.441066+0000","last_peered":"2023-07-06T21:39:26.441066+0000","last_clean":"2023-07-06T21:39:26.441066+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:39:26.441066+0000","last_undegraded":"2023-07-06T21:39:26.441066+0000","last_fullsized":"2023-07-06T21:39:26.441066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:05:58.605808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.437281+0000","last_change":"2023-07-06T21:39:25.955813+0000","last_active":"2023-07-06T21:39:26.437281+0000","last_peered":"2023-07-06T21:39:26.437281+0000","last_clean":"2023-07-06T21:39:26.437281+0000","last_became_active":"2023-07-06T21:39:25.955570+0000","last_became_peered":"2023-07-06T21:39:25.955570+0000","last_unstale":"2023-07-06T21:39:26.437281+0000","last_undegraded":"2023-07-06T21:39:26.437281+0000","last_fullsized":"2023-07-06T21:39:26.437281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465373+0000","last_change":"2023-07-06T21:39:26.458071+0000","last_active":"2023-07-06T21:39:26.465373+0000","last_peered":"2023-07-06T21:39:26.465373+0000","last_clean":"2023-07-06T21:39:26.465373+0000","last_became_active":"2023-07-06T21:39:26.457809+0000","last_became_peered":"2023-07-06T21:39:26.457809+0000","last_unstale":"2023-07-06T21:39:26.465373+0000","last_undegraded":"2023-07-06T21:39:26.465373+0000","last_fullsized":"2023-07-06T21:39:26.465373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:16.841715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.433786+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:39:26.433786+0000","last_peered":"2023-07-06T21:39:26.433786+0000","last_clean":"2023-07-06T21:39:26.433786+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:39:26.433786+0000","last_undegraded":"2023-07-06T21:39:26.433786+0000","last_fullsized":"2023-07-06T21:39:26.433786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:58:39.143427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.446223+0000","last_change":"2023-07-06T21:39:25.951522+0000","last_active":"2023-07-06T21:39:26.446223+0000","last_peered":"2023-07-06T21:39:26.446223+0000","last_clean":"2023-07-06T21:39:26.446223+0000","last_became_active":"2023-07-06T21:39:25.951221+0000","last_became_peered":"2023-07-06T21:39:25.951221+0000","last_unstale":"2023-07-06T21:39:26.446223+0000","last_undegraded":"2023-07-06T21:39:26.446223+0000","last_fullsized":"2023-07-06T21:39:26.446223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465082+0000","last_change":"2023-07-06T21:39:26.456883+0000","last_active":"2023-07-06T21:39:26.465082+0000","last_peered":"2023-07-06T21:39:26.465082+0000","last_clean":"2023-07-06T21:39:26.465082+0000","last_became_active":"2023-07-06T21:39:26.456380+0000","last_became_peered":"2023-07-06T21:39:26.456380+0000","last_unstale":"2023-07-06T21:39:26.465082+0000","last_undegraded":"2023-07-06T21:39:26.465082+0000","last_fullsized":"2023-07-06T21:39:26.465082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.444316+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:39:26.444316+0000","last_peered":"2023-07-06T21:39:26.444316+0000","last_clean":"2023-07-06T21:39:26.444316+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:39:26.444316+0000","last_undegraded":"2023-07-06T21:39:26.444316+0000","last_fullsized":"2023-07-06T21:39:26.444316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:29.856040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.455723+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:39:26.455723+0000","last_peered":"2023-07-06T21:39:26.455723+0000","last_clean":"2023-07-06T21:39:26.455723+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:39:26.455723+0000","last_undegraded":"2023-07-06T21:39:26.455723+0000","last_fullsized":"2023-07-06T21:39:26.455723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:01.411128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.444363+0000","last_change":"2023-07-06T21:39:25.953072+0000","last_active":"2023-07-06T21:39:26.444363+0000","last_peered":"2023-07-06T21:39:26.444363+0000","last_clean":"2023-07-06T21:39:26.444363+0000","last_became_active":"2023-07-06T21:39:25.952723+0000","last_became_peered":"2023-07-06T21:39:25.952723+0000","last_unstale":"2023-07-06T21:39:26.444363+0000","last_undegraded":"2023-07-06T21:39:26.444363+0000","last_fullsized":"2023-07-06T21:39:26.444363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:58.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464390+0000","last_change":"2023-07-06T21:39:26.459006+0000","last_active":"2023-07-06T21:39:26.464390+0000","last_peered":"2023-07-06T21:39:26.464390+0000","last_clean":"2023-07-06T21:39:26.464390+0000","last_became_active":"2023-07-06T21:39:26.458537+0000","last_became_peered":"2023-07-06T21:39:26.458537+0000","last_unstale":"2023-07-06T21:39:26.464390+0000","last_undegraded":"2023-07-06T21:39:26.464390+0000","last_fullsized":"2023-07-06T21:39:26.464390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.456726+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:39:26.456726+0000","last_peered":"2023-07-06T21:39:26.456726+0000","last_clean":"2023-07-06T21:39:26.456726+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:39:26.456726+0000","last_undegraded":"2023-07-06T21:39:26.456726+0000","last_fullsized":"2023-07-06T21:39:26.456726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:07.704609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.443067+0000","last_change":"2023-07-06T21:39:25.951755+0000","last_active":"2023-07-06T21:39:26.443067+0000","last_peered":"2023-07-06T21:39:26.443067+0000","last_clean":"2023-07-06T21:39:26.443067+0000","last_became_active":"2023-07-06T21:39:25.951412+0000","last_became_peered":"2023-07-06T21:39:25.951412+0000","last_unstale":"2023-07-06T21:39:26.443067+0000","last_undegraded":"2023-07-06T21:39:26.443067+0000","last_fullsized":"2023-07-06T21:39:26.443067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:58.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462362+0000","last_change":"2023-07-06T21:39:26.455874+0000","last_active":"2023-07-06T21:39:26.462362+0000","last_peered":"2023-07-06T21:39:26.462362+0000","last_clean":"2023-07-06T21:39:26.462362+0000","last_became_active":"2023-07-06T21:39:26.455628+0000","last_became_peered":"2023-07-06T21:39:26.455628+0000","last_unstale":"2023-07-06T21:39:26.462362+0000","last_undegraded":"2023-07-06T21:39:26.462362+0000","last_fullsized":"2023-07-06T21:39:26.462362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.456330+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:39:26.456330+0000","last_peered":"2023-07-06T21:39:26.456330+0000","last_clean":"2023-07-06T21:39:26.456330+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:39:26.456330+0000","last_undegraded":"2023-07-06T21:39:26.456330+0000","last_fullsized":"2023-07-06T21:39:26.456330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:47.125791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447901+0000","last_change":"2023-07-06T21:39:25.959332+0000","last_active":"2023-07-06T21:39:26.447901+0000","last_peered":"2023-07-06T21:39:26.447901+0000","last_clean":"2023-07-06T21:39:26.447901+0000","last_became_active":"2023-07-06T21:39:25.958983+0000","last_became_peered":"2023-07-06T21:39:25.958983+0000","last_unstale":"2023-07-06T21:39:26.447901+0000","last_undegraded":"2023-07-06T21:39:26.447901+0000","last_fullsized":"2023-07-06T21:39:26.447901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:56.789261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462449+0000","last_change":"2023-07-06T21:39:26.455969+0000","last_active":"2023-07-06T21:39:26.462449+0000","last_peered":"2023-07-06T21:39:26.462449+0000","last_clean":"2023-07-06T21:39:26.462449+0000","last_became_active":"2023-07-06T21:39:26.455716+0000","last_became_peered":"2023-07-06T21:39:26.455716+0000","last_unstale":"2023-07-06T21:39:26.462449+0000","last_undegraded":"2023-07-06T21:39:26.462449+0000","last_fullsized":"2023-07-06T21:39:26.462449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466398+0000","last_change":"2023-07-06T21:39:25.962443+0000","last_active":"2023-07-06T21:39:26.466398+0000","last_peered":"2023-07-06T21:39:26.466398+0000","last_clean":"2023-07-06T21:39:26.466398+0000","last_became_active":"2023-07-06T21:39:25.962123+0000","last_became_peered":"2023-07-06T21:39:25.962123+0000","last_unstale":"2023-07-06T21:39:26.466398+0000","last_undegraded":"2023-07-06T21:39:26.466398+0000","last_fullsized":"2023-07-06T21:39:26.466398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:15.833771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447235+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:39:26.447235+0000","last_peered":"2023-07-06T21:39:26.447235+0000","last_clean":"2023-07-06T21:39:26.447235+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:39:26.447235+0000","last_undegraded":"2023-07-06T21:39:26.447235+0000","last_fullsized":"2023-07-06T21:39:26.447235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:00.879415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465913+0000","last_change":"2023-07-06T21:39:26.463271+0000","last_active":"2023-07-06T21:39:26.465913+0000","last_peered":"2023-07-06T21:39:26.465913+0000","last_clean":"2023-07-06T21:39:26.465913+0000","last_became_active":"2023-07-06T21:39:26.463082+0000","last_became_peered":"2023-07-06T21:39:26.463082+0000","last_unstale":"2023-07-06T21:39:26.465913+0000","last_undegraded":"2023-07-06T21:39:26.465913+0000","last_fullsized":"2023-07-06T21:39:26.465913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.450074+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:39:26.450074+0000","last_peered":"2023-07-06T21:39:26.450074+0000","last_clean":"2023-07-06T21:39:26.450074+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:39:26.450074+0000","last_undegraded":"2023-07-06T21:39:26.450074+0000","last_fullsized":"2023-07-06T21:39:26.450074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:12.312730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442772+0000","last_change":"2023-07-06T21:39:25.946579+0000","last_active":"2023-07-06T21:39:26.442772+0000","last_peered":"2023-07-06T21:39:26.442772+0000","last_clean":"2023-07-06T21:39:26.442772+0000","last_became_active":"2023-07-06T21:39:25.945265+0000","last_became_peered":"2023-07-06T21:39:25.945265+0000","last_unstale":"2023-07-06T21:39:26.442772+0000","last_undegraded":"2023-07-06T21:39:26.442772+0000","last_fullsized":"2023-07-06T21:39:26.442772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:17.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461318+0000","last_change":"2023-07-06T21:39:26.458730+0000","last_active":"2023-07-06T21:39:26.461318+0000","last_peered":"2023-07-06T21:39:26.461318+0000","last_clean":"2023-07-06T21:39:26.461318+0000","last_became_active":"2023-07-06T21:39:26.458438+0000","last_became_peered":"2023-07-06T21:39:26.458438+0000","last_unstale":"2023-07-06T21:39:26.461318+0000","last_undegraded":"2023-07-06T21:39:26.461318+0000","last_fullsized":"2023-07-06T21:39:26.461318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.458858+0000","last_change":"2023-07-06T21:39:25.948372+0000","last_active":"2023-07-06T21:39:26.458858+0000","last_peered":"2023-07-06T21:39:26.458858+0000","last_clean":"2023-07-06T21:39:26.458858+0000","last_became_active":"2023-07-06T21:39:25.948140+0000","last_became_peered":"2023-07-06T21:39:25.948140+0000","last_unstale":"2023-07-06T21:39:26.458858+0000","last_undegraded":"2023-07-06T21:39:26.458858+0000","last_fullsized":"2023-07-06T21:39:26.458858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:04.024304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461215+0000","last_change":"2023-07-06T21:39:26.454713+0000","last_active":"2023-07-06T21:39:26.461215+0000","last_peered":"2023-07-06T21:39:26.461215+0000","last_clean":"2023-07-06T21:39:26.461215+0000","last_became_active":"2023-07-06T21:39:26.454273+0000","last_became_peered":"2023-07-06T21:39:26.454273+0000","last_unstale":"2023-07-06T21:39:26.461215+0000","last_undegraded":"2023-07-06T21:39:26.461215+0000","last_fullsized":"2023-07-06T21:39:26.461215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.460671+0000","last_change":"2023-07-06T21:39:25.951864+0000","last_active":"2023-07-06T21:39:26.460671+0000","last_peered":"2023-07-06T21:39:26.460671+0000","last_clean":"2023-07-06T21:39:26.460671+0000","last_became_active":"2023-07-06T21:39:25.951527+0000","last_became_peered":"2023-07-06T21:39:25.951527+0000","last_unstale":"2023-07-06T21:39:26.460671+0000","last_undegraded":"2023-07-06T21:39:26.460671+0000","last_fullsized":"2023-07-06T21:39:26.460671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:06.297911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457197+0000","last_change":"2023-07-06T21:39:26.450910+0000","last_active":"2023-07-06T21:39:26.457197+0000","last_peered":"2023-07-06T21:39:26.457197+0000","last_clean":"2023-07-06T21:39:26.457197+0000","last_became_active":"2023-07-06T21:39:26.450613+0000","last_became_peered":"2023-07-06T21:39:26.450613+0000","last_unstale":"2023-07-06T21:39:26.457197+0000","last_undegraded":"2023-07-06T21:39:26.457197+0000","last_fullsized":"2023-07-06T21:39:26.457197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:18.739171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.460415+0000","last_change":"2023-07-06T21:39:25.945875+0000","last_active":"2023-07-06T21:39:26.460415+0000","last_peered":"2023-07-06T21:39:26.460415+0000","last_clean":"2023-07-06T21:39:26.460415+0000","last_became_active":"2023-07-06T21:39:25.945533+0000","last_became_peered":"2023-07-06T21:39:25.945533+0000","last_unstale":"2023-07-06T21:39:26.460415+0000","last_undegraded":"2023-07-06T21:39:26.460415+0000","last_fullsized":"2023-07-06T21:39:26.460415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457374+0000","last_change":"2023-07-06T21:39:26.447102+0000","last_active":"2023-07-06T21:39:26.457374+0000","last_peered":"2023-07-06T21:39:26.457374+0000","last_clean":"2023-07-06T21:39:26.457374+0000","last_became_active":"2023-07-06T21:39:26.446750+0000","last_became_peered":"2023-07-06T21:39:26.446750+0000","last_unstale":"2023-07-06T21:39:26.457374+0000","last_undegraded":"2023-07-06T21:39:26.457374+0000","last_fullsized":"2023-07-06T21:39:26.457374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468909+0000","last_change":"2023-07-06T21:39:25.952004+0000","last_active":"2023-07-06T21:39:26.468909+0000","last_peered":"2023-07-06T21:39:26.468909+0000","last_clean":"2023-07-06T21:39:26.468909+0000","last_became_active":"2023-07-06T21:39:25.951708+0000","last_became_peered":"2023-07-06T21:39:25.951708+0000","last_unstale":"2023-07-06T21:39:26.468909+0000","last_undegraded":"2023-07-06T21:39:26.468909+0000","last_fullsized":"2023-07-06T21:39:26.468909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:04.024304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462563+0000","last_change":"2023-07-06T21:39:26.450232+0000","last_active":"2023-07-06T21:39:26.462563+0000","last_peered":"2023-07-06T21:39:26.462563+0000","last_clean":"2023-07-06T21:39:26.462563+0000","last_became_active":"2023-07-06T21:39:26.449627+0000","last_became_peered":"2023-07-06T21:39:26.449627+0000","last_unstale":"2023-07-06T21:39:26.462563+0000","last_undegraded":"2023-07-06T21:39:26.462563+0000","last_fullsized":"2023-07-06T21:39:26.462563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:28.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464773+0000","last_change":"2023-07-06T21:39:25.945282+0000","last_active":"2023-07-06T21:39:26.464773+0000","last_peered":"2023-07-06T21:39:26.464773+0000","last_clean":"2023-07-06T21:39:26.464773+0000","last_became_active":"2023-07-06T21:39:25.944898+0000","last_became_peered":"2023-07-06T21:39:25.944898+0000","last_unstale":"2023-07-06T21:39:26.464773+0000","last_undegraded":"2023-07-06T21:39:26.464773+0000","last_fullsized":"2023-07-06T21:39:26.464773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:32.946775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473692+0000","last_change":"2023-07-06T21:39:26.470499+0000","last_active":"2023-07-06T21:39:26.473692+0000","last_peered":"2023-07-06T21:39:26.473692+0000","last_clean":"2023-07-06T21:39:26.473692+0000","last_became_active":"2023-07-06T21:39:26.470270+0000","last_became_peered":"2023-07-06T21:39:26.470270+0000","last_unstale":"2023-07-06T21:39:26.473692+0000","last_undegraded":"2023-07-06T21:39:26.473692+0000","last_fullsized":"2023-07-06T21:39:26.473692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:58:45.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473603+0000","last_change":"2023-07-06T21:39:26.469954+0000","last_active":"2023-07-06T21:39:26.473603+0000","last_peered":"2023-07-06T21:39:26.473603+0000","last_clean":"2023-07-06T21:39:26.473603+0000","last_became_active":"2023-07-06T21:39:26.469618+0000","last_became_peered":"2023-07-06T21:39:26.469618+0000","last_unstale":"2023-07-06T21:39:26.473603+0000","last_undegraded":"2023-07-06T21:39:26.473603+0000","last_fullsized":"2023-07-06T21:39:26.473603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462583+0000","last_change":"2023-07-06T21:39:26.450370+0000","last_active":"2023-07-06T21:39:26.462583+0000","last_peered":"2023-07-06T21:39:26.462583+0000","last_clean":"2023-07-06T21:39:26.462583+0000","last_became_active":"2023-07-06T21:39:26.449980+0000","last_became_peered":"2023-07-06T21:39:26.449980+0000","last_unstale":"2023-07-06T21:39:26.462583+0000","last_undegraded":"2023-07-06T21:39:26.462583+0000","last_fullsized":"2023-07-06T21:39:26.462583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473600+0000","last_change":"2023-07-06T21:39:26.469630+0000","last_active":"2023-07-06T21:39:26.473600+0000","last_peered":"2023-07-06T21:39:26.473600+0000","last_clean":"2023-07-06T21:39:26.473600+0000","last_became_active":"2023-07-06T21:39:26.469358+0000","last_became_peered":"2023-07-06T21:39:26.469358+0000","last_unstale":"2023-07-06T21:39:26.473600+0000","last_undegraded":"2023-07-06T21:39:26.473600+0000","last_fullsized":"2023-07-06T21:39:26.473600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:15.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470680+0000","last_change":"2023-07-06T21:39:26.466945+0000","last_active":"2023-07-06T21:39:26.470680+0000","last_peered":"2023-07-06T21:39:26.470680+0000","last_clean":"2023-07-06T21:39:26.470680+0000","last_became_active":"2023-07-06T21:39:26.466613+0000","last_became_peered":"2023-07-06T21:39:26.466613+0000","last_unstale":"2023-07-06T21:39:26.470680+0000","last_undegraded":"2023-07-06T21:39:26.470680+0000","last_fullsized":"2023-07-06T21:39:26.470680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468116+0000","last_change":"2023-07-06T21:39:26.466793+0000","last_active":"2023-07-06T21:39:26.468116+0000","last_peered":"2023-07-06T21:39:26.468116+0000","last_clean":"2023-07-06T21:39:26.468116+0000","last_became_active":"2023-07-06T21:39:26.466585+0000","last_became_peered":"2023-07-06T21:39:26.466585+0000","last_unstale":"2023-07-06T21:39:26.468116+0000","last_undegraded":"2023-07-06T21:39:26.468116+0000","last_fullsized":"2023-07-06T21:39:26.468116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:05.032248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":851968,"data_stored":675094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":672877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":83036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":84566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":83598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:39:31.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:39:31.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.505+0000 7feb543ff700 1 Processor -- start 2023-07-06T21:39:31.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.506+0000 7feb543ff700 1 -- start start 2023-07-06T21:39:31.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb543ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0b83c0 0x7feb4c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb543ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb4c0cfc60 0x7feb4c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb543ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0d0a80 con 0x7feb4c0d3710 2023-07-06T21:39:31.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb543ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c0b8790 con 0x7feb4c0cfc60 2023-07-06T21:39:31.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb543ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0b4e70 con 0x7feb4c0b83c0 2023-07-06T21:39:31.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb5199a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7feb4c0d3710 0x7feb4c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52100/0 (socket says 172.21.15.84:52100) 2023-07-06T21:39:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb5199a700 1 -- 172.21.15.84:0/568720807 learned_addr learned my addr 172.21.15.84:0/568720807 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb5299c700 1 --2- 172.21.15.84:0/568720807 >> v2:172.21.15.84:3300/0 conn(0x7feb4c0cfc60 0x7feb4c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb5219b700 1 --2- 172.21.15.84:0/568720807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0b83c0 0x7feb4c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.507+0000 7feb51199700 1 -- 172.21.15.84:0/568720807 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293437903 0 0) 0x7feb4c0d0a80 con 0x7feb4c0d3710 2023-07-06T21:39:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb51199700 1 -- 172.21.15.84:0/568720807 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb380035b0 con 0x7feb4c0d3710 2023-07-06T21:39:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb5299c700 1 -- 172.21.15.84:0/568720807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0b83c0 msgr2=0x7feb4c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb5299c700 1 --2- 172.21.15.84:0/568720807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0b83c0 0x7feb4c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb5299c700 1 -- 172.21.15.84:0/568720807 >> v1:172.21.15.84:6789/0 conn(0x7feb4c0d3710 legacy=0x7feb4c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb5299c700 1 -- 172.21.15.84:0/568720807 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0b5150 con 0x7feb4c0cfc60 2023-07-06T21:39:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb5299c700 1 --2- 172.21.15.84:0/568720807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feb44009d90 tx=0x7feb4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cad0df6987961 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb51199700 1 -- 172.21.15.84:0/568720807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4400e070 con 0x7feb4c0cfc60 2023-07-06T21:39:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb51199700 1 -- 172.21.15.84:0/568720807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4400bb20 con 0x7feb4c0cfc60 2023-07-06T21:39:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.508+0000 7feb51199700 1 -- 172.21.15.84:0/568720807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb44012430 con 0x7feb4c0cfc60 2023-07-06T21:39:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 -- 172.21.15.84:0/568720807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 msgr2=0x7feb4c0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 --2- 172.21.15.84:0/568720807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7feb44009d90 tx=0x7feb4400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 -- 172.21.15.84:0/568720807 shutdown_connections 2023-07-06T21:39:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 --2- 172.21.15.84:0/568720807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0f10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 --2- 172.21.15.84:0/568720807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0b83c0 0x7feb4c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 -- 172.21.15.84:0/568720807 >> 172.21.15.84:0/568720807 conn(0x7feb4c06b350 msgr2=0x7feb4c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 -- 172.21.15.84:0/568720807 shutdown_connections 2023-07-06T21:39:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.509+0000 7feb543ff700 1 -- 172.21.15.84:0/568720807 wait complete. 2023-07-06T21:39:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.511+0000 7feb543ff700 1 Processor -- start 2023-07-06T21:39:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.511+0000 7feb543ff700 1 -- start start 2023-07-06T21:39:31.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb543ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb4c0b83c0 0x7feb4c0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb543ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb543ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0d3710 0x7feb4c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb543ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0c96b0 con 0x7feb4c0cfc60 2023-07-06T21:39:31.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb543ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0b2710 con 0x7feb4c0b83c0 2023-07-06T21:39:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb543ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c0b29f0 con 0x7feb4c0d3710 2023-07-06T21:39:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38454/0 (socket says 172.21.15.84:38454) 2023-07-06T21:39:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 -- 172.21.15.84:0/2793008141 learned_addr learned my addr 172.21.15.84:0/2793008141 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5219b700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb4c0b83c0 0x7feb4c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5299c700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0d3710 0x7feb4c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 -- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0d3710 msgr2=0x7feb4c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0d3710 0x7feb4c0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 -- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb4c0b83c0 msgr2=0x7feb4c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb4c0b83c0 0x7feb4c0b3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.513+0000 7feb5199a700 1 -- 172.21.15.84:0/2793008141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44009a70 con 0x7feb4c0cfc60 2023-07-06T21:39:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.514+0000 7feb5199a700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb3c005910 tx=0x7feb3c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.514+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c005d70 con 0x7feb4c0cfc60 2023-07-06T21:39:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.514+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c0070d0 con 0x7feb4c0cfc60 2023-07-06T21:39:31.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.514+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c0b2cd0 con 0x7feb4c0cfc60 2023-07-06T21:39:31.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.514+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c007460 con 0x7feb4c0cfc60 2023-07-06T21:39:31.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.514+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c0b4410 con 0x7feb4c0cfc60 2023-07-06T21:39:31.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.515+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c0b7b80 con 0x7feb4c0cfc60 2023-07-06T21:39:31.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.516+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb3c0075c0 con 0x7feb4c0cfc60 2023-07-06T21:39:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.517+0000 7feb42ffd700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb3006ace0 0x7feb3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.517+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb3c0888b0 con 0x7feb4c0cfc60 2023-07-06T21:39:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.517+0000 7feb5219b700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb3006ace0 0x7feb3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.518+0000 7feb5219b700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb3006ace0 0x7feb3006d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0b4850 tx=0x7feb48009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.521+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb3c053950 con 0x7feb4c0cfc60 2023-07-06T21:39:31.708 INFO:tasks.ceph.mds.d.smithi107.stderr:2023-07-06T21:39:31.705+0000 7f4edf9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T21:39:31.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.712+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb4c0b2b60 con 0x7feb3006ace0 2023-07-06T21:39:31.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.720+0000 7feb42ffd700 1 -- 172.21.15.84:0/2793008141 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286666 (secure 0 0 0) 0x7feb4c0b2b60 con 0x7feb3006ace0 2023-07-06T21:39:31.726 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb3006ace0 msgr2=0x7feb3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb3006ace0 0x7feb3006d190 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feb4c0b4850 tx=0x7feb48009210 comp rx=0 tx=0).stop 2023-07-06T21:39:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 msgr2=0x7feb4c0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0e40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7feb3c005910 tx=0x7feb3c00c9e0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 shutdown_connections 2023-07-06T21:39:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb3006ace0 0x7feb3006d190 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb4c0d3710 0x7feb4c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb4c0b83c0 0x7feb4c0b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 --2- 172.21.15.84:0/2793008141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb4c0cfc60 0x7feb4c0d0e40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.724+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 >> 172.21.15.84:0/2793008141 conn(0x7feb4c06b350 msgr2=0x7feb4c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.725+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 shutdown_connections 2023-07-06T21:39:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.725+0000 7feb543ff700 1 -- 172.21.15.84:0/2793008141 wait complete. 2023-07-06T21:39:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:39:31.746 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-07-06T21:39:30.122797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218628,"kb_used_data":3012,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754756092,"statfs":{"total":773094113280,"available":772870238208,"internally_reserved":0,"allocated":3084288,"data_stored":1848009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.198367"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459444+0000","last_change":"2023-07-06T21:39:26.450219+0000","last_active":"2023-07-06T21:39:26.459444+0000","last_peered":"2023-07-06T21:39:26.459444+0000","last_clean":"2023-07-06T21:39:26.459444+0000","last_became_active":"2023-07-06T21:39:26.449158+0000","last_became_peered":"2023-07-06T21:39:26.449158+0000","last_unstale":"2023-07-06T21:39:26.459444+0000","last_undegraded":"2023-07-06T21:39:26.459444+0000","last_fullsized":"2023-07-06T21:39:26.459444+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464823+0000","last_change":"2023-07-06T21:39:26.456425+0000","last_active":"2023-07-06T21:39:26.464823+0000","last_peered":"2023-07-06T21:39:26.464823+0000","last_clean":"2023-07-06T21:39:26.464823+0000","last_became_active":"2023-07-06T21:39:26.455881+0000","last_became_peered":"2023-07-06T21:39:26.455881+0000","last_unstale":"2023-07-06T21:39:26.464823+0000","last_undegraded":"2023-07-06T21:39:26.464823+0000","last_fullsized":"2023-07-06T21:39:26.464823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:57.797206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457614+0000","last_change":"2023-07-06T21:39:26.448957+0000","last_active":"2023-07-06T21:39:26.457614+0000","last_peered":"2023-07-06T21:39:26.457614+0000","last_clean":"2023-07-06T21:39:26.457614+0000","last_became_active":"2023-07-06T21:39:26.448647+0000","last_became_peered":"2023-07-06T21:39:26.448647+0000","last_unstale":"2023-07-06T21:39:26.457614+0000","last_undegraded":"2023-07-06T21:39:26.457614+0000","last_fullsized":"2023-07-06T21:39:26.457614+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:16.841715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470422+0000","last_change":"2023-07-06T21:39:26.468418+0000","last_active":"2023-07-06T21:39:26.470422+0000","last_peered":"2023-07-06T21:39:26.470422+0000","last_clean":"2023-07-06T21:39:26.470422+0000","last_became_active":"2023-07-06T21:39:26.464060+0000","last_became_peered":"2023-07-06T21:39:26.464060+0000","last_unstale":"2023-07-06T21:39:26.470422+0000","last_undegraded":"2023-07-06T21:39:26.470422+0000","last_fullsized":"2023-07-06T21:39:26.470422+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465353+0000","last_change":"2023-07-06T21:39:26.461152+0000","last_active":"2023-07-06T21:39:26.465353+0000","last_peered":"2023-07-06T21:39:26.465353+0000","last_clean":"2023-07-06T21:39:26.465353+0000","last_became_active":"2023-07-06T21:39:26.460854+0000","last_became_peered":"2023-07-06T21:39:26.460854+0000","last_unstale":"2023-07-06T21:39:26.465353+0000","last_undegraded":"2023-07-06T21:39:26.465353+0000","last_fullsized":"2023-07-06T21:39:26.465353+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457397+0000","last_change":"2023-07-06T21:39:26.449460+0000","last_active":"2023-07-06T21:39:26.457397+0000","last_peered":"2023-07-06T21:39:26.457397+0000","last_clean":"2023-07-06T21:39:26.457397+0000","last_became_active":"2023-07-06T21:39:26.449145+0000","last_became_peered":"2023-07-06T21:39:26.449145+0000","last_unstale":"2023-07-06T21:39:26.457397+0000","last_undegraded":"2023-07-06T21:39:26.457397+0000","last_fullsized":"2023-07-06T21:39:26.457397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469794+0000","last_change":"2023-07-06T21:39:26.467462+0000","last_active":"2023-07-06T21:39:26.469794+0000","last_peered":"2023-07-06T21:39:26.469794+0000","last_clean":"2023-07-06T21:39:26.469794+0000","last_became_active":"2023-07-06T21:39:26.466827+0000","last_became_peered":"2023-07-06T21:39:26.466827+0000","last_unstale":"2023-07-06T21:39:26.469794+0000","last_undegraded":"2023-07-06T21:39:26.469794+0000","last_fullsized":"2023-07-06T21:39:26.469794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:11.539304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.467206+0000","last_change":"2023-07-06T21:39:26.459017+0000","last_active":"2023-07-06T21:39:26.467206+0000","last_peered":"2023-07-06T21:39:26.467206+0000","last_clean":"2023-07-06T21:39:26.467206+0000","last_became_active":"2023-07-06T21:39:26.458749+0000","last_became_peered":"2023-07-06T21:39:26.458749+0000","last_unstale":"2023-07-06T21:39:26.467206+0000","last_undegraded":"2023-07-06T21:39:26.467206+0000","last_fullsized":"2023-07-06T21:39:26.467206+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461280+0000","last_change":"2023-07-06T21:39:26.453377+0000","last_active":"2023-07-06T21:39:26.461280+0000","last_peered":"2023-07-06T21:39:26.461280+0000","last_clean":"2023-07-06T21:39:26.461280+0000","last_became_active":"2023-07-06T21:39:26.452864+0000","last_became_peered":"2023-07-06T21:39:26.452864+0000","last_unstale":"2023-07-06T21:39:26.461280+0000","last_undegraded":"2023-07-06T21:39:26.461280+0000","last_fullsized":"2023-07-06T21:39:26.461280+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464800+0000","last_change":"2023-07-06T21:39:26.456191+0000","last_active":"2023-07-06T21:39:26.464800+0000","last_peered":"2023-07-06T21:39:26.464800+0000","last_clean":"2023-07-06T21:39:26.464800+0000","last_became_active":"2023-07-06T21:39:26.455459+0000","last_became_peered":"2023-07-06T21:39:26.455459+0000","last_unstale":"2023-07-06T21:39:26.464800+0000","last_undegraded":"2023-07-06T21:39:26.464800+0000","last_fullsized":"2023-07-06T21:39:26.464800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:48.843759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457676+0000","last_change":"2023-07-06T21:39:26.450168+0000","last_active":"2023-07-06T21:39:26.457676+0000","last_peered":"2023-07-06T21:39:26.457676+0000","last_clean":"2023-07-06T21:39:26.457676+0000","last_became_active":"2023-07-06T21:39:26.449224+0000","last_became_peered":"2023-07-06T21:39:26.449224+0000","last_unstale":"2023-07-06T21:39:26.457676+0000","last_undegraded":"2023-07-06T21:39:26.457676+0000","last_fullsized":"2023-07-06T21:39:26.457676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461192+0000","last_change":"2023-07-06T21:39:26.454525+0000","last_active":"2023-07-06T21:39:26.461192+0000","last_peered":"2023-07-06T21:39:26.461192+0000","last_clean":"2023-07-06T21:39:26.461192+0000","last_became_active":"2023-07-06T21:39:26.453872+0000","last_became_peered":"2023-07-06T21:39:26.453872+0000","last_unstale":"2023-07-06T21:39:26.461192+0000","last_undegraded":"2023-07-06T21:39:26.461192+0000","last_fullsized":"2023-07-06T21:39:26.461192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459328+0000","last_change":"2023-07-06T21:39:26.449670+0000","last_active":"2023-07-06T21:39:26.459328+0000","last_peered":"2023-07-06T21:39:26.459328+0000","last_clean":"2023-07-06T21:39:26.459328+0000","last_became_active":"2023-07-06T21:39:26.449116+0000","last_became_peered":"2023-07-06T21:39:26.449116+0000","last_unstale":"2023-07-06T21:39:26.459328+0000","last_undegraded":"2023-07-06T21:39:26.459328+0000","last_fullsized":"2023-07-06T21:39:26.459328+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:33.107986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.460744+0000","last_change":"2023-07-06T21:39:26.450117+0000","last_active":"2023-07-06T21:39:26.460744+0000","last_peered":"2023-07-06T21:39:26.460744+0000","last_clean":"2023-07-06T21:39:26.460744+0000","last_became_active":"2023-07-06T21:39:26.449800+0000","last_became_peered":"2023-07-06T21:39:26.449800+0000","last_unstale":"2023-07-06T21:39:26.460744+0000","last_undegraded":"2023-07-06T21:39:26.460744+0000","last_fullsized":"2023-07-06T21:39:26.460744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469812+0000","last_change":"2023-07-06T21:39:26.467633+0000","last_active":"2023-07-06T21:39:26.469812+0000","last_peered":"2023-07-06T21:39:26.469812+0000","last_clean":"2023-07-06T21:39:26.469812+0000","last_became_active":"2023-07-06T21:39:26.467194+0000","last_became_peered":"2023-07-06T21:39:26.467194+0000","last_unstale":"2023-07-06T21:39:26.469812+0000","last_undegraded":"2023-07-06T21:39:26.469812+0000","last_fullsized":"2023-07-06T21:39:26.469812+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:48.843759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459075+0000","last_change":"2023-07-06T21:39:26.448432+0000","last_active":"2023-07-06T21:39:26.459075+0000","last_peered":"2023-07-06T21:39:26.459075+0000","last_clean":"2023-07-06T21:39:26.459075+0000","last_became_active":"2023-07-06T21:39:26.448091+0000","last_became_peered":"2023-07-06T21:39:26.448091+0000","last_unstale":"2023-07-06T21:39:26.459075+0000","last_undegraded":"2023-07-06T21:39:26.459075+0000","last_fullsized":"2023-07-06T21:39:26.459075+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:28.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461260+0000","last_change":"2023-07-06T21:39:26.453156+0000","last_active":"2023-07-06T21:39:26.461260+0000","last_peered":"2023-07-06T21:39:26.461260+0000","last_clean":"2023-07-06T21:39:26.461260+0000","last_became_active":"2023-07-06T21:39:26.452499+0000","last_became_peered":"2023-07-06T21:39:26.452499+0000","last_unstale":"2023-07-06T21:39:26.461260+0000","last_undegraded":"2023-07-06T21:39:26.461260+0000","last_fullsized":"2023-07-06T21:39:26.461260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:33.107986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.454139+0000","last_change":"2023-07-06T21:39:26.442470+0000","last_active":"2023-07-06T21:39:26.454139+0000","last_peered":"2023-07-06T21:39:26.454139+0000","last_clean":"2023-07-06T21:39:26.454139+0000","last_became_active":"2023-07-06T21:39:26.442162+0000","last_became_peered":"2023-07-06T21:39:26.442162+0000","last_unstale":"2023-07-06T21:39:26.454139+0000","last_undegraded":"2023-07-06T21:39:26.454139+0000","last_fullsized":"2023-07-06T21:39:26.454139+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473217+0000","last_change":"2023-07-06T21:39:26.469782+0000","last_active":"2023-07-06T21:39:26.473217+0000","last_peered":"2023-07-06T21:39:26.473217+0000","last_clean":"2023-07-06T21:39:26.473217+0000","last_became_active":"2023-07-06T21:39:26.469492+0000","last_became_peered":"2023-07-06T21:39:26.469492+0000","last_unstale":"2023-07-06T21:39:26.473217+0000","last_undegraded":"2023-07-06T21:39:26.473217+0000","last_fullsized":"2023-07-06T21:39:26.473217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.472585+0000","last_change":"2023-07-06T21:39:26.465478+0000","last_active":"2023-07-06T21:39:26.472585+0000","last_peered":"2023-07-06T21:39:26.472585+0000","last_clean":"2023-07-06T21:39:26.472585+0000","last_became_active":"2023-07-06T21:39:26.465171+0000","last_became_peered":"2023-07-06T21:39:26.465171+0000","last_unstale":"2023-07-06T21:39:26.472585+0000","last_undegraded":"2023-07-06T21:39:26.472585+0000","last_fullsized":"2023-07-06T21:39:26.472585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:05.032248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470274+0000","last_change":"2023-07-06T21:39:26.465956+0000","last_active":"2023-07-06T21:39:26.470274+0000","last_peered":"2023-07-06T21:39:26.470274+0000","last_clean":"2023-07-06T21:39:26.470274+0000","last_became_active":"2023-07-06T21:39:26.465672+0000","last_became_peered":"2023-07-06T21:39:26.465672+0000","last_unstale":"2023-07-06T21:39:26.470274+0000","last_undegraded":"2023-07-06T21:39:26.470274+0000","last_fullsized":"2023-07-06T21:39:26.470274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:28.800563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461210+0000","last_change":"2023-07-06T21:39:26.453279+0000","last_active":"2023-07-06T21:39:26.461210+0000","last_peered":"2023-07-06T21:39:26.461210+0000","last_clean":"2023-07-06T21:39:26.461210+0000","last_became_active":"2023-07-06T21:39:26.452832+0000","last_became_peered":"2023-07-06T21:39:26.452832+0000","last_unstale":"2023-07-06T21:39:26.461210+0000","last_undegraded":"2023-07-06T21:39:26.461210+0000","last_fullsized":"2023-07-06T21:39:26.461210+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:58:45.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462541+0000","last_change":"2023-07-06T21:39:26.450178+0000","last_active":"2023-07-06T21:39:26.462541+0000","last_peered":"2023-07-06T21:39:26.462541+0000","last_clean":"2023-07-06T21:39:26.462541+0000","last_became_active":"2023-07-06T21:39:26.449355+0000","last_became_peered":"2023-07-06T21:39:26.449355+0000","last_unstale":"2023-07-06T21:39:26.462541+0000","last_undegraded":"2023-07-06T21:39:26.462541+0000","last_fullsized":"2023-07-06T21:39:26.462541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.467846+0000","last_change":"2023-07-06T21:39:26.465115+0000","last_active":"2023-07-06T21:39:26.467846+0000","last_peered":"2023-07-06T21:39:26.467846+0000","last_clean":"2023-07-06T21:39:26.467846+0000","last_became_active":"2023-07-06T21:39:26.464450+0000","last_became_peered":"2023-07-06T21:39:26.464450+0000","last_unstale":"2023-07-06T21:39:26.467846+0000","last_undegraded":"2023-07-06T21:39:26.467846+0000","last_fullsized":"2023-07-06T21:39:26.467846+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466037+0000","last_change":"2023-07-06T21:39:26.464834+0000","last_active":"2023-07-06T21:39:26.466037+0000","last_peered":"2023-07-06T21:39:26.466037+0000","last_clean":"2023-07-06T21:39:26.466037+0000","last_became_active":"2023-07-06T21:39:26.464519+0000","last_became_peered":"2023-07-06T21:39:26.464519+0000","last_unstale":"2023-07-06T21:39:26.466037+0000","last_undegraded":"2023-07-06T21:39:26.466037+0000","last_fullsized":"2023-07-06T21:39:26.466037+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:08.131343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461455+0000","last_change":"2023-07-06T21:39:26.451689+0000","last_active":"2023-07-06T21:39:26.461455+0000","last_peered":"2023-07-06T21:39:26.461455+0000","last_clean":"2023-07-06T21:39:26.461455+0000","last_became_active":"2023-07-06T21:39:26.451397+0000","last_became_peered":"2023-07-06T21:39:26.451397+0000","last_unstale":"2023-07-06T21:39:26.461455+0000","last_undegraded":"2023-07-06T21:39:26.461455+0000","last_fullsized":"2023-07-06T21:39:26.461455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468459+0000","last_change":"2023-07-06T21:39:26.466401+0000","last_active":"2023-07-06T21:39:26.468459+0000","last_peered":"2023-07-06T21:39:26.468459+0000","last_clean":"2023-07-06T21:39:26.468459+0000","last_became_active":"2023-07-06T21:39:26.465938+0000","last_became_peered":"2023-07-06T21:39:26.465938+0000","last_unstale":"2023-07-06T21:39:26.468459+0000","last_undegraded":"2023-07-06T21:39:26.468459+0000","last_fullsized":"2023-07-06T21:39:26.468459+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457230+0000","last_change":"2023-07-06T21:39:25.954109+0000","last_active":"2023-07-06T21:39:26.457230+0000","last_peered":"2023-07-06T21:39:26.457230+0000","last_clean":"2023-07-06T21:39:26.457230+0000","last_became_active":"2023-07-06T21:39:25.953788+0000","last_became_peered":"2023-07-06T21:39:25.953788+0000","last_unstale":"2023-07-06T21:39:26.457230+0000","last_undegraded":"2023-07-06T21:39:26.457230+0000","last_fullsized":"2023-07-06T21:39:26.457230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462717+0000","last_change":"2023-07-06T21:39:26.455786+0000","last_active":"2023-07-06T21:39:26.462717+0000","last_peered":"2023-07-06T21:39:26.462717+0000","last_clean":"2023-07-06T21:39:26.462717+0000","last_became_active":"2023-07-06T21:39:26.455415+0000","last_became_peered":"2023-07-06T21:39:26.455415+0000","last_unstale":"2023-07-06T21:39:26.462717+0000","last_undegraded":"2023-07-06T21:39:26.462717+0000","last_fullsized":"2023-07-06T21:39:26.462717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:18.739171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.446240+0000","last_change":"2023-07-06T21:39:25.952238+0000","last_active":"2023-07-06T21:39:26.446240+0000","last_peered":"2023-07-06T21:39:26.446240+0000","last_clean":"2023-07-06T21:39:26.446240+0000","last_became_active":"2023-07-06T21:39:25.951890+0000","last_became_peered":"2023-07-06T21:39:25.951890+0000","last_unstale":"2023-07-06T21:39:26.446240+0000","last_undegraded":"2023-07-06T21:39:26.446240+0000","last_fullsized":"2023-07-06T21:39:26.446240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465983+0000","last_change":"2023-07-06T21:39:26.463846+0000","last_active":"2023-07-06T21:39:26.465983+0000","last_peered":"2023-07-06T21:39:26.465983+0000","last_clean":"2023-07-06T21:39:26.465983+0000","last_became_active":"2023-07-06T21:39:26.463530+0000","last_became_peered":"2023-07-06T21:39:26.463530+0000","last_unstale":"2023-07-06T21:39:26.465983+0000","last_undegraded":"2023-07-06T21:39:26.465983+0000","last_fullsized":"2023-07-06T21:39:26.465983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447565+0000","last_change":"2023-07-06T21:39:25.958386+0000","last_active":"2023-07-06T21:39:26.447565+0000","last_peered":"2023-07-06T21:39:26.447565+0000","last_clean":"2023-07-06T21:39:26.447565+0000","last_became_active":"2023-07-06T21:39:25.958049+0000","last_became_peered":"2023-07-06T21:39:25.958049+0000","last_unstale":"2023-07-06T21:39:26.447565+0000","last_undegraded":"2023-07-06T21:39:26.447565+0000","last_fullsized":"2023-07-06T21:39:26.447565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.463897+0000","last_change":"2023-07-06T21:39:26.461047+0000","last_active":"2023-07-06T21:39:26.463897+0000","last_peered":"2023-07-06T21:39:26.463897+0000","last_clean":"2023-07-06T21:39:26.463897+0000","last_became_active":"2023-07-06T21:39:26.460608+0000","last_became_peered":"2023-07-06T21:39:26.460608+0000","last_unstale":"2023-07-06T21:39:26.463897+0000","last_undegraded":"2023-07-06T21:39:26.463897+0000","last_fullsized":"2023-07-06T21:39:26.463897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442814+0000","last_change":"2023-07-06T21:39:25.953273+0000","last_active":"2023-07-06T21:39:26.442814+0000","last_peered":"2023-07-06T21:39:26.442814+0000","last_clean":"2023-07-06T21:39:26.442814+0000","last_became_active":"2023-07-06T21:39:25.947023+0000","last_became_peered":"2023-07-06T21:39:25.947023+0000","last_unstale":"2023-07-06T21:39:26.442814+0000","last_undegraded":"2023-07-06T21:39:26.442814+0000","last_fullsized":"2023-07-06T21:39:26.442814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:43.182084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462635+0000","last_change":"2023-07-06T21:39:26.453860+0000","last_active":"2023-07-06T21:39:26.462635+0000","last_peered":"2023-07-06T21:39:26.462635+0000","last_clean":"2023-07-06T21:39:26.462635+0000","last_became_active":"2023-07-06T21:39:26.453597+0000","last_became_peered":"2023-07-06T21:39:26.453597+0000","last_unstale":"2023-07-06T21:39:26.462635+0000","last_undegraded":"2023-07-06T21:39:26.462635+0000","last_fullsized":"2023-07-06T21:39:26.462635+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:15.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.443836+0000","last_change":"2023-07-06T21:39:25.949777+0000","last_active":"2023-07-06T21:39:26.443836+0000","last_peered":"2023-07-06T21:39:26.443836+0000","last_clean":"2023-07-06T21:39:26.443836+0000","last_became_active":"2023-07-06T21:39:25.949379+0000","last_became_peered":"2023-07-06T21:39:25.949379+0000","last_unstale":"2023-07-06T21:39:26.443836+0000","last_undegraded":"2023-07-06T21:39:26.443836+0000","last_fullsized":"2023-07-06T21:39:26.443836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:10.672168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465950+0000","last_change":"2023-07-06T21:39:26.463815+0000","last_active":"2023-07-06T21:39:26.465950+0000","last_peered":"2023-07-06T21:39:26.465950+0000","last_clean":"2023-07-06T21:39:26.465950+0000","last_became_active":"2023-07-06T21:39:26.463557+0000","last_became_peered":"2023-07-06T21:39:26.463557+0000","last_unstale":"2023-07-06T21:39:26.465950+0000","last_undegraded":"2023-07-06T21:39:26.465950+0000","last_fullsized":"2023-07-06T21:39:26.465950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466327+0000","last_change":"2023-07-06T21:39:25.963442+0000","last_active":"2023-07-06T21:39:26.466327+0000","last_peered":"2023-07-06T21:39:26.466327+0000","last_clean":"2023-07-06T21:39:26.466327+0000","last_became_active":"2023-07-06T21:39:25.963058+0000","last_became_peered":"2023-07-06T21:39:25.963058+0000","last_unstale":"2023-07-06T21:39:26.466327+0000","last_undegraded":"2023-07-06T21:39:26.466327+0000","last_fullsized":"2023-07-06T21:39:26.466327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:10.672168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465705+0000","last_change":"2023-07-06T21:39:26.464284+0000","last_active":"2023-07-06T21:39:26.465705+0000","last_peered":"2023-07-06T21:39:26.465705+0000","last_clean":"2023-07-06T21:39:26.465705+0000","last_became_active":"2023-07-06T21:39:26.463874+0000","last_became_peered":"2023-07-06T21:39:26.463874+0000","last_unstale":"2023-07-06T21:39:26.465705+0000","last_undegraded":"2023-07-06T21:39:26.465705+0000","last_fullsized":"2023-07-06T21:39:26.465705+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.474140+0000","last_change":"2023-07-06T21:39:25.950225+0000","last_active":"2023-07-06T21:39:26.474140+0000","last_peered":"2023-07-06T21:39:26.474140+0000","last_clean":"2023-07-06T21:39:26.474140+0000","last_became_active":"2023-07-06T21:39:25.949924+0000","last_became_peered":"2023-07-06T21:39:25.949924+0000","last_unstale":"2023-07-06T21:39:26.474140+0000","last_undegraded":"2023-07-06T21:39:26.474140+0000","last_fullsized":"2023-07-06T21:39:26.474140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:06.297911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465101+0000","last_change":"2023-07-06T21:39:26.463354+0000","last_active":"2023-07-06T21:39:26.465101+0000","last_peered":"2023-07-06T21:39:26.465101+0000","last_clean":"2023-07-06T21:39:26.465101+0000","last_became_active":"2023-07-06T21:39:26.463039+0000","last_became_peered":"2023-07-06T21:39:26.463039+0000","last_unstale":"2023-07-06T21:39:26.465101+0000","last_undegraded":"2023-07-06T21:39:26.465101+0000","last_fullsized":"2023-07-06T21:39:26.465101+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:33.107986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469446+0000","last_change":"2023-07-06T21:39:25.951134+0000","last_active":"2023-07-06T21:39:26.469446+0000","last_peered":"2023-07-06T21:39:26.469446+0000","last_clean":"2023-07-06T21:39:26.469446+0000","last_became_active":"2023-07-06T21:39:25.950821+0000","last_became_peered":"2023-07-06T21:39:25.950821+0000","last_unstale":"2023-07-06T21:39:26.469446+0000","last_undegraded":"2023-07-06T21:39:26.469446+0000","last_fullsized":"2023-07-06T21:39:26.469446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462736+0000","last_change":"2023-07-06T21:39:26.455921+0000","last_active":"2023-07-06T21:39:26.462736+0000","last_peered":"2023-07-06T21:39:26.462736+0000","last_clean":"2023-07-06T21:39:26.462736+0000","last_became_active":"2023-07-06T21:39:26.455574+0000","last_became_peered":"2023-07-06T21:39:26.455574+0000","last_unstale":"2023-07-06T21:39:26.462736+0000","last_undegraded":"2023-07-06T21:39:26.462736+0000","last_fullsized":"2023-07-06T21:39:26.462736+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442838+0000","last_change":"2023-07-06T21:39:25.949803+0000","last_active":"2023-07-06T21:39:26.442838+0000","last_peered":"2023-07-06T21:39:26.442838+0000","last_clean":"2023-07-06T21:39:26.442838+0000","last_became_active":"2023-07-06T21:39:25.949490+0000","last_became_peered":"2023-07-06T21:39:25.949490+0000","last_unstale":"2023-07-06T21:39:26.442838+0000","last_undegraded":"2023-07-06T21:39:26.442838+0000","last_fullsized":"2023-07-06T21:39:26.442838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:38.093027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468440+0000","last_change":"2023-07-06T21:39:26.466204+0000","last_active":"2023-07-06T21:39:26.468440+0000","last_peered":"2023-07-06T21:39:26.468440+0000","last_clean":"2023-07-06T21:39:26.468440+0000","last_became_active":"2023-07-06T21:39:26.465576+0000","last_became_peered":"2023-07-06T21:39:26.465576+0000","last_unstale":"2023-07-06T21:39:26.468440+0000","last_undegraded":"2023-07-06T21:39:26.468440+0000","last_fullsized":"2023-07-06T21:39:26.468440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:16.841715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459462+0000","last_change":"2023-07-06T21:39:25.952231+0000","last_active":"2023-07-06T21:39:26.459462+0000","last_peered":"2023-07-06T21:39:26.459462+0000","last_clean":"2023-07-06T21:39:26.459462+0000","last_became_active":"2023-07-06T21:39:25.951945+0000","last_became_peered":"2023-07-06T21:39:25.951945+0000","last_unstale":"2023-07-06T21:39:26.459462+0000","last_undegraded":"2023-07-06T21:39:26.459462+0000","last_fullsized":"2023-07-06T21:39:26.459462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:55.145873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465391+0000","last_change":"2023-07-06T21:39:26.460570+0000","last_active":"2023-07-06T21:39:26.465391+0000","last_peered":"2023-07-06T21:39:26.465391+0000","last_clean":"2023-07-06T21:39:26.465391+0000","last_became_active":"2023-07-06T21:39:26.460310+0000","last_became_peered":"2023-07-06T21:39:26.460310+0000","last_unstale":"2023-07-06T21:39:26.465391+0000","last_undegraded":"2023-07-06T21:39:26.465391+0000","last_fullsized":"2023-07-06T21:39:26.465391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:28.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.469415+0000","last_change":"2023-07-06T21:39:25.953216+0000","last_active":"2023-07-06T21:39:26.469415+0000","last_peered":"2023-07-06T21:39:26.469415+0000","last_clean":"2023-07-06T21:39:26.469415+0000","last_became_active":"2023-07-06T21:39:25.952808+0000","last_became_peered":"2023-07-06T21:39:25.952808+0000","last_unstale":"2023-07-06T21:39:26.469415+0000","last_undegraded":"2023-07-06T21:39:26.469415+0000","last_fullsized":"2023-07-06T21:39:26.469415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:14.829725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465918+0000","last_change":"2023-07-06T21:39:26.463251+0000","last_active":"2023-07-06T21:39:26.465918+0000","last_peered":"2023-07-06T21:39:26.465918+0000","last_clean":"2023-07-06T21:39:26.465918+0000","last_became_active":"2023-07-06T21:39:26.463003+0000","last_became_peered":"2023-07-06T21:39:26.463003+0000","last_unstale":"2023-07-06T21:39:26.465918+0000","last_undegraded":"2023-07-06T21:39:26.465918+0000","last_fullsized":"2023-07-06T21:39:26.465918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462670+0000","last_change":"2023-07-06T21:39:25.950786+0000","last_active":"2023-07-06T21:39:26.462670+0000","last_peered":"2023-07-06T21:39:26.462670+0000","last_clean":"2023-07-06T21:39:26.462670+0000","last_became_active":"2023-07-06T21:39:25.950367+0000","last_became_peered":"2023-07-06T21:39:25.950367+0000","last_unstale":"2023-07-06T21:39:26.462670+0000","last_undegraded":"2023-07-06T21:39:26.462670+0000","last_fullsized":"2023-07-06T21:39:26.462670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:56.789261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.467393+0000","last_change":"2023-07-06T21:39:26.465718+0000","last_active":"2023-07-06T21:39:26.467393+0000","last_peered":"2023-07-06T21:39:26.467393+0000","last_clean":"2023-07-06T21:39:26.467393+0000","last_became_active":"2023-07-06T21:39:26.465470+0000","last_became_peered":"2023-07-06T21:39:26.465470+0000","last_unstale":"2023-07-06T21:39:26.467393+0000","last_undegraded":"2023-07-06T21:39:26.467393+0000","last_fullsized":"2023-07-06T21:39:26.467393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462581+0000","last_change":"2023-07-06T21:39:25.950737+0000","last_active":"2023-07-06T21:39:26.462581+0000","last_peered":"2023-07-06T21:39:26.462581+0000","last_clean":"2023-07-06T21:39:26.462581+0000","last_became_active":"2023-07-06T21:39:25.950252+0000","last_became_peered":"2023-07-06T21:39:25.950252+0000","last_unstale":"2023-07-06T21:39:26.462581+0000","last_undegraded":"2023-07-06T21:39:26.462581+0000","last_fullsized":"2023-07-06T21:39:26.462581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:06.297911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457222+0000","last_change":"2023-07-06T21:39:26.449765+0000","last_active":"2023-07-06T21:39:26.457222+0000","last_peered":"2023-07-06T21:39:26.457222+0000","last_clean":"2023-07-06T21:39:26.457222+0000","last_became_active":"2023-07-06T21:39:26.449453+0000","last_became_peered":"2023-07-06T21:39:26.449453+0000","last_unstale":"2023-07-06T21:39:26.457222+0000","last_undegraded":"2023-07-06T21:39:26.457222+0000","last_fullsized":"2023-07-06T21:39:26.457222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468835+0000","last_change":"2023-07-06T21:39:25.951600+0000","last_active":"2023-07-06T21:39:26.468835+0000","last_peered":"2023-07-06T21:39:26.468835+0000","last_clean":"2023-07-06T21:39:26.468835+0000","last_became_active":"2023-07-06T21:39:25.951237+0000","last_became_peered":"2023-07-06T21:39:25.951237+0000","last_unstale":"2023-07-06T21:39:26.468835+0000","last_undegraded":"2023-07-06T21:39:26.468835+0000","last_fullsized":"2023-07-06T21:39:26.468835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:35.274536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470358+0000","last_change":"2023-07-06T21:39:26.466709+0000","last_active":"2023-07-06T21:39:26.470358+0000","last_peered":"2023-07-06T21:39:26.470358+0000","last_clean":"2023-07-06T21:39:26.470358+0000","last_became_active":"2023-07-06T21:39:26.463332+0000","last_became_peered":"2023-07-06T21:39:26.463332+0000","last_unstale":"2023-07-06T21:39:26.470358+0000","last_undegraded":"2023-07-06T21:39:26.470358+0000","last_fullsized":"2023-07-06T21:39:26.470358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442593+0000","last_change":"2023-07-06T21:39:25.953896+0000","last_active":"2023-07-06T21:39:26.442593+0000","last_peered":"2023-07-06T21:39:26.442593+0000","last_clean":"2023-07-06T21:39:26.442593+0000","last_became_active":"2023-07-06T21:39:25.953607+0000","last_became_peered":"2023-07-06T21:39:25.953607+0000","last_unstale":"2023-07-06T21:39:26.442593+0000","last_undegraded":"2023-07-06T21:39:26.442593+0000","last_fullsized":"2023-07-06T21:39:26.442593+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461226+0000","last_change":"2023-07-06T21:39:26.453327+0000","last_active":"2023-07-06T21:39:26.461226+0000","last_peered":"2023-07-06T21:39:26.461226+0000","last_clean":"2023-07-06T21:39:26.461226+0000","last_became_active":"2023-07-06T21:39:26.453059+0000","last_became_peered":"2023-07-06T21:39:26.453059+0000","last_unstale":"2023-07-06T21:39:26.461226+0000","last_undegraded":"2023-07-06T21:39:26.461226+0000","last_fullsized":"2023-07-06T21:39:26.461226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:15.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466101+0000","last_change":"2023-07-06T21:39:25.955876+0000","last_active":"2023-07-06T21:39:26.466101+0000","last_peered":"2023-07-06T21:39:26.466101+0000","last_clean":"2023-07-06T21:39:26.466101+0000","last_became_active":"2023-07-06T21:39:25.955527+0000","last_became_peered":"2023-07-06T21:39:25.955527+0000","last_unstale":"2023-07-06T21:39:26.466101+0000","last_undegraded":"2023-07-06T21:39:26.466101+0000","last_fullsized":"2023-07-06T21:39:26.466101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:32.946775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.459049+0000","last_change":"2023-07-06T21:39:26.452205+0000","last_active":"2023-07-06T21:39:26.459049+0000","last_peered":"2023-07-06T21:39:26.459049+0000","last_clean":"2023-07-06T21:39:26.459049+0000","last_became_active":"2023-07-06T21:39:26.451938+0000","last_became_peered":"2023-07-06T21:39:26.451938+0000","last_unstale":"2023-07-06T21:39:26.459049+0000","last_undegraded":"2023-07-06T21:39:26.459049+0000","last_fullsized":"2023-07-06T21:39:26.459049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447669+0000","last_change":"2023-07-06T21:39:25.954596+0000","last_active":"2023-07-06T21:39:26.447669+0000","last_peered":"2023-07-06T21:39:26.447669+0000","last_clean":"2023-07-06T21:39:26.447669+0000","last_became_active":"2023-07-06T21:39:25.954316+0000","last_became_peered":"2023-07-06T21:39:25.954316+0000","last_unstale":"2023-07-06T21:39:26.447669+0000","last_undegraded":"2023-07-06T21:39:26.447669+0000","last_fullsized":"2023-07-06T21:39:26.447669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:27.268839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457592+0000","last_change":"2023-07-06T21:39:26.446371+0000","last_active":"2023-07-06T21:39:26.457592+0000","last_peered":"2023-07-06T21:39:26.457592+0000","last_clean":"2023-07-06T21:39:26.457592+0000","last_became_active":"2023-07-06T21:39:26.445294+0000","last_became_peered":"2023-07-06T21:39:26.445294+0000","last_unstale":"2023-07-06T21:39:26.457592+0000","last_undegraded":"2023-07-06T21:39:26.457592+0000","last_fullsized":"2023-07-06T21:39:26.457592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.437948+0000","last_change":"2023-07-06T21:39:25.954319+0000","last_active":"2023-07-06T21:39:26.437948+0000","last_peered":"2023-07-06T21:39:26.437948+0000","last_clean":"2023-07-06T21:39:26.437948+0000","last_became_active":"2023-07-06T21:39:25.954089+0000","last_became_peered":"2023-07-06T21:39:25.954089+0000","last_unstale":"2023-07-06T21:39:26.437948+0000","last_undegraded":"2023-07-06T21:39:26.437948+0000","last_fullsized":"2023-07-06T21:39:26.437948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465097+0000","last_change":"2023-07-06T21:39:26.463246+0000","last_active":"2023-07-06T21:39:26.465097+0000","last_peered":"2023-07-06T21:39:26.465097+0000","last_clean":"2023-07-06T21:39:26.465097+0000","last_became_active":"2023-07-06T21:39:26.462881+0000","last_became_peered":"2023-07-06T21:39:26.462881+0000","last_unstale":"2023-07-06T21:39:26.465097+0000","last_undegraded":"2023-07-06T21:39:26.465097+0000","last_fullsized":"2023-07-06T21:39:26.465097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447412+0000","last_change":"2023-07-06T21:39:25.962410+0000","last_active":"2023-07-06T21:39:26.447412+0000","last_peered":"2023-07-06T21:39:26.447412+0000","last_clean":"2023-07-06T21:39:26.447412+0000","last_became_active":"2023-07-06T21:39:25.962150+0000","last_became_peered":"2023-07-06T21:39:25.962150+0000","last_unstale":"2023-07-06T21:39:26.447412+0000","last_undegraded":"2023-07-06T21:39:26.447412+0000","last_fullsized":"2023-07-06T21:39:26.447412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.456964+0000","last_change":"2023-07-06T21:39:26.449718+0000","last_active":"2023-07-06T21:39:26.456964+0000","last_peered":"2023-07-06T21:39:26.456964+0000","last_clean":"2023-07-06T21:39:26.456964+0000","last_became_active":"2023-07-06T21:39:26.449444+0000","last_became_peered":"2023-07-06T21:39:26.449444+0000","last_unstale":"2023-07-06T21:39:26.456964+0000","last_undegraded":"2023-07-06T21:39:26.456964+0000","last_fullsized":"2023-07-06T21:39:26.456964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:18.739171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447717+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:39:26.447717+0000","last_peered":"2023-07-06T21:39:26.447717+0000","last_clean":"2023-07-06T21:39:26.447717+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:39:26.447717+0000","last_undegraded":"2023-07-06T21:39:26.447717+0000","last_fullsized":"2023-07-06T21:39:26.447717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:42.417319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442354+0000","last_change":"2023-07-06T21:39:25.962785+0000","last_active":"2023-07-06T21:39:26.442354+0000","last_peered":"2023-07-06T21:39:26.442354+0000","last_clean":"2023-07-06T21:39:26.442354+0000","last_became_active":"2023-07-06T21:39:25.962456+0000","last_became_peered":"2023-07-06T21:39:25.962456+0000","last_unstale":"2023-07-06T21:39:26.442354+0000","last_undegraded":"2023-07-06T21:39:26.442354+0000","last_fullsized":"2023-07-06T21:39:26.442354+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465217+0000","last_change":"2023-07-06T21:39:26.457017+0000","last_active":"2023-07-06T21:39:26.465217+0000","last_peered":"2023-07-06T21:39:26.465217+0000","last_clean":"2023-07-06T21:39:26.465217+0000","last_became_active":"2023-07-06T21:39:26.456767+0000","last_became_peered":"2023-07-06T21:39:26.456767+0000","last_unstale":"2023-07-06T21:39:26.465217+0000","last_undegraded":"2023-07-06T21:39:26.465217+0000","last_fullsized":"2023-07-06T21:39:26.465217+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.441066+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:39:26.441066+0000","last_peered":"2023-07-06T21:39:26.441066+0000","last_clean":"2023-07-06T21:39:26.441066+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:39:26.441066+0000","last_undegraded":"2023-07-06T21:39:26.441066+0000","last_fullsized":"2023-07-06T21:39:26.441066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:05:58.605808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.437281+0000","last_change":"2023-07-06T21:39:25.955813+0000","last_active":"2023-07-06T21:39:26.437281+0000","last_peered":"2023-07-06T21:39:26.437281+0000","last_clean":"2023-07-06T21:39:26.437281+0000","last_became_active":"2023-07-06T21:39:25.955570+0000","last_became_peered":"2023-07-06T21:39:25.955570+0000","last_unstale":"2023-07-06T21:39:26.437281+0000","last_undegraded":"2023-07-06T21:39:26.437281+0000","last_fullsized":"2023-07-06T21:39:26.437281+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465373+0000","last_change":"2023-07-06T21:39:26.458071+0000","last_active":"2023-07-06T21:39:26.465373+0000","last_peered":"2023-07-06T21:39:26.465373+0000","last_clean":"2023-07-06T21:39:26.465373+0000","last_became_active":"2023-07-06T21:39:26.457809+0000","last_became_peered":"2023-07-06T21:39:26.457809+0000","last_unstale":"2023-07-06T21:39:26.465373+0000","last_undegraded":"2023-07-06T21:39:26.465373+0000","last_fullsized":"2023-07-06T21:39:26.465373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:16.841715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.433786+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:39:26.433786+0000","last_peered":"2023-07-06T21:39:26.433786+0000","last_clean":"2023-07-06T21:39:26.433786+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:39:26.433786+0000","last_undegraded":"2023-07-06T21:39:26.433786+0000","last_fullsized":"2023-07-06T21:39:26.433786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:58:39.143427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.446223+0000","last_change":"2023-07-06T21:39:25.951522+0000","last_active":"2023-07-06T21:39:26.446223+0000","last_peered":"2023-07-06T21:39:26.446223+0000","last_clean":"2023-07-06T21:39:26.446223+0000","last_became_active":"2023-07-06T21:39:25.951221+0000","last_became_peered":"2023-07-06T21:39:25.951221+0000","last_unstale":"2023-07-06T21:39:26.446223+0000","last_undegraded":"2023-07-06T21:39:26.446223+0000","last_fullsized":"2023-07-06T21:39:26.446223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465082+0000","last_change":"2023-07-06T21:39:26.456883+0000","last_active":"2023-07-06T21:39:26.465082+0000","last_peered":"2023-07-06T21:39:26.465082+0000","last_clean":"2023-07-06T21:39:26.465082+0000","last_became_active":"2023-07-06T21:39:26.456380+0000","last_became_peered":"2023-07-06T21:39:26.456380+0000","last_unstale":"2023-07-06T21:39:26.465082+0000","last_undegraded":"2023-07-06T21:39:26.465082+0000","last_fullsized":"2023-07-06T21:39:26.465082+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.444316+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:39:26.444316+0000","last_peered":"2023-07-06T21:39:26.444316+0000","last_clean":"2023-07-06T21:39:26.444316+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:39:26.444316+0000","last_undegraded":"2023-07-06T21:39:26.444316+0000","last_fullsized":"2023-07-06T21:39:26.444316+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:29.856040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":173,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.455723+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:39:26.455723+0000","last_peered":"2023-07-06T21:39:26.455723+0000","last_clean":"2023-07-06T21:39:26.455723+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:39:26.455723+0000","last_undegraded":"2023-07-06T21:39:26.455723+0000","last_fullsized":"2023-07-06T21:39:26.455723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:01.411128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.444363+0000","last_change":"2023-07-06T21:39:25.953072+0000","last_active":"2023-07-06T21:39:26.444363+0000","last_peered":"2023-07-06T21:39:26.444363+0000","last_clean":"2023-07-06T21:39:26.444363+0000","last_became_active":"2023-07-06T21:39:25.952723+0000","last_became_peered":"2023-07-06T21:39:25.952723+0000","last_unstale":"2023-07-06T21:39:26.444363+0000","last_undegraded":"2023-07-06T21:39:26.444363+0000","last_fullsized":"2023-07-06T21:39:26.444363+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:58.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464390+0000","last_change":"2023-07-06T21:39:26.459006+0000","last_active":"2023-07-06T21:39:26.464390+0000","last_peered":"2023-07-06T21:39:26.464390+0000","last_clean":"2023-07-06T21:39:26.464390+0000","last_became_active":"2023-07-06T21:39:26.458537+0000","last_became_peered":"2023-07-06T21:39:26.458537+0000","last_unstale":"2023-07-06T21:39:26.464390+0000","last_undegraded":"2023-07-06T21:39:26.464390+0000","last_fullsized":"2023-07-06T21:39:26.464390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:15.864363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.456726+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:39:26.456726+0000","last_peered":"2023-07-06T21:39:26.456726+0000","last_clean":"2023-07-06T21:39:26.456726+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:39:26.456726+0000","last_undegraded":"2023-07-06T21:39:26.456726+0000","last_fullsized":"2023-07-06T21:39:26.456726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:07.704609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.443067+0000","last_change":"2023-07-06T21:39:25.951755+0000","last_active":"2023-07-06T21:39:26.443067+0000","last_peered":"2023-07-06T21:39:26.443067+0000","last_clean":"2023-07-06T21:39:26.443067+0000","last_became_active":"2023-07-06T21:39:25.951412+0000","last_became_peered":"2023-07-06T21:39:25.951412+0000","last_unstale":"2023-07-06T21:39:26.443067+0000","last_undegraded":"2023-07-06T21:39:26.443067+0000","last_fullsized":"2023-07-06T21:39:26.443067+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:58.200772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462362+0000","last_change":"2023-07-06T21:39:26.455874+0000","last_active":"2023-07-06T21:39:26.462362+0000","last_peered":"2023-07-06T21:39:26.462362+0000","last_clean":"2023-07-06T21:39:26.462362+0000","last_became_active":"2023-07-06T21:39:26.455628+0000","last_became_peered":"2023-07-06T21:39:26.455628+0000","last_unstale":"2023-07-06T21:39:26.462362+0000","last_undegraded":"2023-07-06T21:39:26.462362+0000","last_fullsized":"2023-07-06T21:39:26.462362+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.456330+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:39:26.456330+0000","last_peered":"2023-07-06T21:39:26.456330+0000","last_clean":"2023-07-06T21:39:26.456330+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:39:26.456330+0000","last_undegraded":"2023-07-06T21:39:26.456330+0000","last_fullsized":"2023-07-06T21:39:26.456330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:47.125791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447901+0000","last_change":"2023-07-06T21:39:25.959332+0000","last_active":"2023-07-06T21:39:26.447901+0000","last_peered":"2023-07-06T21:39:26.447901+0000","last_clean":"2023-07-06T21:39:26.447901+0000","last_became_active":"2023-07-06T21:39:25.958983+0000","last_became_peered":"2023-07-06T21:39:25.958983+0000","last_unstale":"2023-07-06T21:39:26.447901+0000","last_undegraded":"2023-07-06T21:39:26.447901+0000","last_fullsized":"2023-07-06T21:39:26.447901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:56.789261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462449+0000","last_change":"2023-07-06T21:39:26.455969+0000","last_active":"2023-07-06T21:39:26.462449+0000","last_peered":"2023-07-06T21:39:26.462449+0000","last_clean":"2023-07-06T21:39:26.462449+0000","last_became_active":"2023-07-06T21:39:26.455716+0000","last_became_peered":"2023-07-06T21:39:26.455716+0000","last_unstale":"2023-07-06T21:39:26.462449+0000","last_undegraded":"2023-07-06T21:39:26.462449+0000","last_fullsized":"2023-07-06T21:39:26.462449+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.466398+0000","last_change":"2023-07-06T21:39:25.962443+0000","last_active":"2023-07-06T21:39:26.466398+0000","last_peered":"2023-07-06T21:39:26.466398+0000","last_clean":"2023-07-06T21:39:26.466398+0000","last_became_active":"2023-07-06T21:39:25.962123+0000","last_became_peered":"2023-07-06T21:39:25.962123+0000","last_unstale":"2023-07-06T21:39:26.466398+0000","last_undegraded":"2023-07-06T21:39:26.466398+0000","last_fullsized":"2023-07-06T21:39:26.466398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:15.833771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.447235+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:39:26.447235+0000","last_peered":"2023-07-06T21:39:26.447235+0000","last_clean":"2023-07-06T21:39:26.447235+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:39:26.447235+0000","last_undegraded":"2023-07-06T21:39:26.447235+0000","last_fullsized":"2023-07-06T21:39:26.447235+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:00.879415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.465913+0000","last_change":"2023-07-06T21:39:26.463271+0000","last_active":"2023-07-06T21:39:26.465913+0000","last_peered":"2023-07-06T21:39:26.465913+0000","last_clean":"2023-07-06T21:39:26.465913+0000","last_became_active":"2023-07-06T21:39:26.463082+0000","last_became_peered":"2023-07-06T21:39:26.463082+0000","last_unstale":"2023-07-06T21:39:26.465913+0000","last_undegraded":"2023-07-06T21:39:26.465913+0000","last_fullsized":"2023-07-06T21:39:26.465913+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.450074+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:39:26.450074+0000","last_peered":"2023-07-06T21:39:26.450074+0000","last_clean":"2023-07-06T21:39:26.450074+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:39:26.450074+0000","last_undegraded":"2023-07-06T21:39:26.450074+0000","last_fullsized":"2023-07-06T21:39:26.450074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:12.312730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.442772+0000","last_change":"2023-07-06T21:39:25.946579+0000","last_active":"2023-07-06T21:39:26.442772+0000","last_peered":"2023-07-06T21:39:26.442772+0000","last_clean":"2023-07-06T21:39:26.442772+0000","last_became_active":"2023-07-06T21:39:25.945265+0000","last_became_peered":"2023-07-06T21:39:25.945265+0000","last_unstale":"2023-07-06T21:39:26.442772+0000","last_undegraded":"2023-07-06T21:39:26.442772+0000","last_fullsized":"2023-07-06T21:39:26.442772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:17.731226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461318+0000","last_change":"2023-07-06T21:39:26.458730+0000","last_active":"2023-07-06T21:39:26.461318+0000","last_peered":"2023-07-06T21:39:26.461318+0000","last_clean":"2023-07-06T21:39:26.461318+0000","last_became_active":"2023-07-06T21:39:26.458438+0000","last_became_peered":"2023-07-06T21:39:26.458438+0000","last_unstale":"2023-07-06T21:39:26.461318+0000","last_undegraded":"2023-07-06T21:39:26.461318+0000","last_fullsized":"2023-07-06T21:39:26.461318+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.458858+0000","last_change":"2023-07-06T21:39:25.948372+0000","last_active":"2023-07-06T21:39:26.458858+0000","last_peered":"2023-07-06T21:39:26.458858+0000","last_clean":"2023-07-06T21:39:26.458858+0000","last_became_active":"2023-07-06T21:39:25.948140+0000","last_became_peered":"2023-07-06T21:39:25.948140+0000","last_unstale":"2023-07-06T21:39:26.458858+0000","last_undegraded":"2023-07-06T21:39:26.458858+0000","last_fullsized":"2023-07-06T21:39:26.458858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:04.024304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.461215+0000","last_change":"2023-07-06T21:39:26.454713+0000","last_active":"2023-07-06T21:39:26.461215+0000","last_peered":"2023-07-06T21:39:26.461215+0000","last_clean":"2023-07-06T21:39:26.461215+0000","last_became_active":"2023-07-06T21:39:26.454273+0000","last_became_peered":"2023-07-06T21:39:26.454273+0000","last_unstale":"2023-07-06T21:39:26.461215+0000","last_undegraded":"2023-07-06T21:39:26.461215+0000","last_fullsized":"2023-07-06T21:39:26.461215+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:14.942421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.460671+0000","last_change":"2023-07-06T21:39:25.951864+0000","last_active":"2023-07-06T21:39:26.460671+0000","last_peered":"2023-07-06T21:39:26.460671+0000","last_clean":"2023-07-06T21:39:26.460671+0000","last_became_active":"2023-07-06T21:39:25.951527+0000","last_became_peered":"2023-07-06T21:39:25.951527+0000","last_unstale":"2023-07-06T21:39:26.460671+0000","last_undegraded":"2023-07-06T21:39:26.460671+0000","last_fullsized":"2023-07-06T21:39:26.460671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:06.297911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457197+0000","last_change":"2023-07-06T21:39:26.450910+0000","last_active":"2023-07-06T21:39:26.457197+0000","last_peered":"2023-07-06T21:39:26.457197+0000","last_clean":"2023-07-06T21:39:26.457197+0000","last_became_active":"2023-07-06T21:39:26.450613+0000","last_became_peered":"2023-07-06T21:39:26.450613+0000","last_unstale":"2023-07-06T21:39:26.457197+0000","last_undegraded":"2023-07-06T21:39:26.457197+0000","last_fullsized":"2023-07-06T21:39:26.457197+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:18.739171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.460415+0000","last_change":"2023-07-06T21:39:25.945875+0000","last_active":"2023-07-06T21:39:26.460415+0000","last_peered":"2023-07-06T21:39:26.460415+0000","last_clean":"2023-07-06T21:39:26.460415+0000","last_became_active":"2023-07-06T21:39:25.945533+0000","last_became_peered":"2023-07-06T21:39:25.945533+0000","last_unstale":"2023-07-06T21:39:26.460415+0000","last_undegraded":"2023-07-06T21:39:26.460415+0000","last_fullsized":"2023-07-06T21:39:26.460415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.457374+0000","last_change":"2023-07-06T21:39:26.447102+0000","last_active":"2023-07-06T21:39:26.457374+0000","last_peered":"2023-07-06T21:39:26.457374+0000","last_clean":"2023-07-06T21:39:26.457374+0000","last_became_active":"2023-07-06T21:39:26.446750+0000","last_became_peered":"2023-07-06T21:39:26.446750+0000","last_unstale":"2023-07-06T21:39:26.457374+0000","last_undegraded":"2023-07-06T21:39:26.457374+0000","last_fullsized":"2023-07-06T21:39:26.457374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468909+0000","last_change":"2023-07-06T21:39:25.952004+0000","last_active":"2023-07-06T21:39:26.468909+0000","last_peered":"2023-07-06T21:39:26.468909+0000","last_clean":"2023-07-06T21:39:26.468909+0000","last_became_active":"2023-07-06T21:39:25.951708+0000","last_became_peered":"2023-07-06T21:39:25.951708+0000","last_unstale":"2023-07-06T21:39:26.468909+0000","last_undegraded":"2023-07-06T21:39:26.468909+0000","last_fullsized":"2023-07-06T21:39:26.468909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:04.024304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462563+0000","last_change":"2023-07-06T21:39:26.450232+0000","last_active":"2023-07-06T21:39:26.462563+0000","last_peered":"2023-07-06T21:39:26.462563+0000","last_clean":"2023-07-06T21:39:26.462563+0000","last_became_active":"2023-07-06T21:39:26.449627+0000","last_became_peered":"2023-07-06T21:39:26.449627+0000","last_unstale":"2023-07-06T21:39:26.462563+0000","last_undegraded":"2023-07-06T21:39:26.462563+0000","last_fullsized":"2023-07-06T21:39:26.462563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:38:28.276784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.464773+0000","last_change":"2023-07-06T21:39:25.945282+0000","last_active":"2023-07-06T21:39:26.464773+0000","last_peered":"2023-07-06T21:39:26.464773+0000","last_clean":"2023-07-06T21:39:26.464773+0000","last_became_active":"2023-07-06T21:39:25.944898+0000","last_became_peered":"2023-07-06T21:39:25.944898+0000","last_unstale":"2023-07-06T21:39:26.464773+0000","last_undegraded":"2023-07-06T21:39:26.464773+0000","last_fullsized":"2023-07-06T21:39:26.464773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:32.946775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473692+0000","last_change":"2023-07-06T21:39:26.470499+0000","last_active":"2023-07-06T21:39:26.473692+0000","last_peered":"2023-07-06T21:39:26.473692+0000","last_clean":"2023-07-06T21:39:26.473692+0000","last_became_active":"2023-07-06T21:39:26.470270+0000","last_became_peered":"2023-07-06T21:39:26.470270+0000","last_unstale":"2023-07-06T21:39:26.473692+0000","last_undegraded":"2023-07-06T21:39:26.473692+0000","last_fullsized":"2023-07-06T21:39:26.473692+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:58:45.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473603+0000","last_change":"2023-07-06T21:39:26.469954+0000","last_active":"2023-07-06T21:39:26.473603+0000","last_peered":"2023-07-06T21:39:26.473603+0000","last_clean":"2023-07-06T21:39:26.473603+0000","last_became_active":"2023-07-06T21:39:26.469618+0000","last_became_peered":"2023-07-06T21:39:26.469618+0000","last_unstale":"2023-07-06T21:39:26.473603+0000","last_undegraded":"2023-07-06T21:39:26.473603+0000","last_fullsized":"2023-07-06T21:39:26.473603+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:57:16.153279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.462583+0000","last_change":"2023-07-06T21:39:26.450370+0000","last_active":"2023-07-06T21:39:26.462583+0000","last_peered":"2023-07-06T21:39:26.462583+0000","last_clean":"2023-07-06T21:39:26.462583+0000","last_became_active":"2023-07-06T21:39:26.449980+0000","last_became_peered":"2023-07-06T21:39:26.449980+0000","last_unstale":"2023-07-06T21:39:26.462583+0000","last_undegraded":"2023-07-06T21:39:26.462583+0000","last_fullsized":"2023-07-06T21:39:26.462583+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:16.280462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.473600+0000","last_change":"2023-07-06T21:39:26.469630+0000","last_active":"2023-07-06T21:39:26.473600+0000","last_peered":"2023-07-06T21:39:26.473600+0000","last_clean":"2023-07-06T21:39:26.473600+0000","last_became_active":"2023-07-06T21:39:26.469358+0000","last_became_peered":"2023-07-06T21:39:26.469358+0000","last_unstale":"2023-07-06T21:39:26.473600+0000","last_undegraded":"2023-07-06T21:39:26.473600+0000","last_fullsized":"2023-07-06T21:39:26.473600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:01:15.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.470680+0000","last_change":"2023-07-06T21:39:26.466945+0000","last_active":"2023-07-06T21:39:26.470680+0000","last_peered":"2023-07-06T21:39:26.470680+0000","last_clean":"2023-07-06T21:39:26.470680+0000","last_became_active":"2023-07-06T21:39:26.466613+0000","last_became_peered":"2023-07-06T21:39:26.466613+0000","last_unstale":"2023-07-06T21:39:26.470680+0000","last_undegraded":"2023-07-06T21:39:26.470680+0000","last_fullsized":"2023-07-06T21:39:26.470680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:39:36.902048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-07-06T21:39:26.468116+0000","last_change":"2023-07-06T21:39:26.466793+0000","last_active":"2023-07-06T21:39:26.468116+0000","last_peered":"2023-07-06T21:39:26.468116+0000","last_clean":"2023-07-06T21:39:26.468116+0000","last_became_active":"2023-07-06T21:39:26.466585+0000","last_became_peered":"2023-07-06T21:39:26.466585+0000","last_unstale":"2023-07-06T21:39:26.468116+0000","last_undegraded":"2023-07-06T21:39:26.468116+0000","last_fullsized":"2023-07-06T21:39:26.468116+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:05.032248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27784,"kb_used_data":832,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344056,"statfs":{"total":96636764160,"available":96608313344,"internally_reserved":0,"allocated":851968,"data_stored":675094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":823296,"data_stored":672877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.748}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":82946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":233472,"data_stored":83036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":84566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.379}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":237568,"data_stored":83598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:39:31.755 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T21:39:31.756 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-06T21:39:31.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-07-06T21:39:31.756 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-07-06T21:39:31.757 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-07-06T21:39:31.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.978+0000 7f3f827ff700 1 Processor -- start 2023-07-06T21:39:31.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.978+0000 7f3f827ff700 1 -- start start 2023-07-06T21:39:31.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f827ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0b83c0 0x7f3f7c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f827ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f7c0cfc60 0x7f3f7c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f827ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7c0d0a80 con 0x7f3f7c0d3710 2023-07-06T21:39:31.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f827ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7c0b8790 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f827ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c0b4e70 con 0x7f3f7c0b83c0 2023-07-06T21:39:31.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f7b7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f7c0d3710 0x7f3f7c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52112/0 (socket says 172.21.15.84:52112) 2023-07-06T21:39:31.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.979+0000 7f3f7b7fe700 1 -- 172.21.15.84:0/2689724850 learned_addr learned my addr 172.21.15.84:0/2689724850 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:31.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f80d9c700 1 --2- 172.21.15.84:0/2689724850 >> v2:172.21.15.107:3300/0 conn(0x7f3f7c0cfc60 0x7f3f7c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f7bfff700 1 --2- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0b83c0 0x7f3f7c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f7affd700 1 -- 172.21.15.84:0/2689724850 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611159786 0 0) 0x7f3f7c0d0a80 con 0x7f3f7c0d3710 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f7affd700 1 -- 172.21.15.84:0/2689724850 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f640035b0 con 0x7f3f7c0d3710 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f80d9c700 1 -- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0b83c0 msgr2=0x7f3f7c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f80d9c700 1 --2- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0b83c0 0x7f3f7c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f80d9c700 1 -- 172.21.15.84:0/2689724850 >> v1:172.21.15.84:6789/0 conn(0x7f3f7c0d3710 legacy=0x7f3f7c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f80d9c700 1 -- 172.21.15.84:0/2689724850 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c0b5150 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.980+0000 7f3f80d9c700 1 --2- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009d90 tx=0x7f3f7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=354551f75efc11fc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f7affd700 1 -- 172.21.15.84:0/2689724850 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7400e070 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f7affd700 1 -- 172.21.15.84:0/2689724850 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7400bb20 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f7affd700 1 -- 172.21.15.84:0/2689724850 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74012430 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 -- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0cfc60 msgr2=0x7f3f7c0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 --2- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009d90 tx=0x7f3f7400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 -- 172.21.15.84:0/2689724850 shutdown_connections 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 --2- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0f10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 --2- 172.21.15.84:0/2689724850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0b83c0 0x7f3f7c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 -- 172.21.15.84:0/2689724850 >> 172.21.15.84:0/2689724850 conn(0x7f3f7c06b350 msgr2=0x7f3f7c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:31.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 -- 172.21.15.84:0/2689724850 shutdown_connections 2023-07-06T21:39:31.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.981+0000 7f3f827ff700 1 -- 172.21.15.84:0/2689724850 wait complete. 2023-07-06T21:39:31.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.983+0000 7f3f827ff700 1 Processor -- start 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.983+0000 7f3f827ff700 1 -- start start 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f827ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0b83c0 0x7f3f7c0b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f827ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f827ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0d3710 0x7f3f7c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f827ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c0c96b0 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f827ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c0b2710 con 0x7f3f7c0b83c0 2023-07-06T21:39:31.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f827ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c0b29f0 con 0x7f3f7c0d3710 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38462/0 (socket says 172.21.15.84:38462) 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 -- 172.21.15.84:0/1312112870 learned_addr learned my addr 172.21.15.84:0/1312112870 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7bfff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0b83c0 0x7f3f7c0b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 -- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0d3710 msgr2=0x7f3f7c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0d3710 0x7f3f7c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 -- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0b83c0 msgr2=0x7f3f7c0b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:31.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.984+0000 7f3f7b7fe700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0b83c0 0x7f3f7c0b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f7b7fe700 1 -- 172.21.15.84:0/1312112870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f74009a70 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f7b7fe700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f70005910 tx=0x7f3f7000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70005d70 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f700070d0 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c0b2cd0 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007460 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.985+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c0b4410 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.986+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c0b7b80 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.987+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f700075c0 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.988+0000 7f3f78ff9700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f6806ace0 0x7f3f6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:31.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.988+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f70088ab0 con 0x7f3f7c0cfc60 2023-07-06T21:39:31.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.988+0000 7f3f7bfff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f6806ace0 0x7f3f6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:31.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.989+0000 7f3f7bfff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f6806ace0 0x7f3f6806d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0b4850 tx=0x7f3f6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:31.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:31.992+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f70053b50 con 0x7f3f7c0cfc60 2023-07-06T21:39:32.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.248+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3f7c073580 con 0x7f3f7c0cfc60 2023-07-06T21:39:32.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.252+0000 7f3f78ff9700 1 -- 172.21.15.84:0/1312112870 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+170841 (secure 0 0 0) 0x7f3f70057170 con 0x7f3f7c0cfc60 2023-07-06T21:39:32.256 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f6806ace0 msgr2=0x7f3f6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f6806ace0 0x7f3f6806d190 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c0b4850 tx=0x7f3f6c008040 comp rx=0 tx=0).stop 2023-07-06T21:39:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 msgr2=0x7f3f7c0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0e40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3f70005910 tx=0x7f3f7000ca90 comp rx=0 tx=0).stop 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 shutdown_connections 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f6806ace0 0x7f3f6806d190 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f7c0d3710 0x7f3f7c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f7c0b83c0 0x7f3f7c0b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 --2- 172.21.15.84:0/1312112870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f7c0cfc60 0x7f3f7c0d0e40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 >> 172.21.15.84:0/1312112870 conn(0x7f3f7c06b350 msgr2=0x7f3f7c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 shutdown_connections 2023-07-06T21:39:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.255+0000 7f3f827ff700 1 -- 172.21.15.84:0/1312112870 wait complete. 2023-07-06T21:39:32.276 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":5,"active_gid":4105,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":4263456437},{"type":"v1","addr":"172.21.15.84:6833","nonce":4263456437}]},"active_addr":"172.21.15.84:6833/4263456437","active_change":"2023-07-06T21:39:06.106866+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":4032291913}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2375957431}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2544901938}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":944180637}]}]} 2023-07-06T21:39:32.280 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-07-06T21:39:32.281 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-07-06T21:39:32.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:32.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.470+0000 7fcd84dff700 1 Processor -- start 2023-07-06T21:39:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.470+0000 7fcd84dff700 1 -- start start 2023-07-06T21:39:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd84dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcd800d3710 0x7fcd800cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd84dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd800cd910 con 0x7fcd800b83c0 2023-07-06T21:39:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd84dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd800d0a80 con 0x7fcd800cfa90 2023-07-06T21:39:32.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd84dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd800cff10 con 0x7fcd800d3710 2023-07-06T21:39:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7e59c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fcd800b83c0 0x7fcd800cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:52126/0 (socket says 172.21.15.84:52126) 2023-07-06T21:39:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7e59c700 1 -- 172.21.15.84:0/3692921521 learned_addr learned my addr 172.21.15.84:0/3692921521 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:32.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7dd9b700 1 --2- 172.21.15.84:0/3692921521 >> v2:172.21.15.84:3300/0 conn(0x7fcd800d3710 0x7fcd800cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7d59a700 1 -- 172.21.15.84:0/3692921521 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822986975 0 0) 0x7fcd70001110 con 0x7fcd800b83c0 2023-07-06T21:39:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7d59a700 1 -- 172.21.15.84:0/3692921521 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd680035b0 con 0x7fcd800b83c0 2023-07-06T21:39:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/3692921521 >> v1:172.21.15.107:6789/0 conn(0x7fcd800cfa90 legacy=0x7fcd800d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/3692921521 >> v1:172.21.15.84:6789/0 conn(0x7fcd800b83c0 legacy=0x7fcd800cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.472+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/3692921521 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800b8790 con 0x7fcd800d3710 2023-07-06T21:39:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd7dd9b700 1 --2- 172.21.15.84:0/3692921521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800d3710 0x7fcd800cf550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009840 tx=0x7fcd74009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c634efc86204039c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd7d59a700 1 -- 172.21.15.84:0/3692921521 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74011830 con 0x7fcd800d3710 2023-07-06T21:39:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd7d59a700 1 -- 172.21.15.84:0/3692921521 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd74011990 con 0x7fcd800d3710 2023-07-06T21:39:32.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd7d59a700 1 -- 172.21.15.84:0/3692921521 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74011d20 con 0x7fcd800d3710 2023-07-06T21:39:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd84dff700 1 -- 172.21.15.84:0/3692921521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800d3710 msgr2=0x7fcd800cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd84dff700 1 --2- 172.21.15.84:0/3692921521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800d3710 0x7fcd800cf550 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009840 tx=0x7fcd74009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd84dff700 1 -- 172.21.15.84:0/3692921521 shutdown_connections 2023-07-06T21:39:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd84dff700 1 --2- 172.21.15.84:0/3692921521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800d3710 0x7fcd800cf550 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.473+0000 7fcd84dff700 1 -- 172.21.15.84:0/3692921521 >> 172.21.15.84:0/3692921521 conn(0x7fcd8006b350 msgr2=0x7fcd8006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.474+0000 7fcd84dff700 1 -- 172.21.15.84:0/3692921521 shutdown_connections 2023-07-06T21:39:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.474+0000 7fcd84dff700 1 -- 172.21.15.84:0/3692921521 wait complete. 2023-07-06T21:39:32.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.475+0000 7fcd84dff700 1 Processor -- start 2023-07-06T21:39:32.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.475+0000 7fcd84dff700 1 -- start start 2023-07-06T21:39:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd84dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd800b83c0 0x7fcd800c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd84dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 0x7fcd80150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd84dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd800d3710 0x7fcd800cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd84dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800c6c50 con 0x7fcd800cfa90 2023-07-06T21:39:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd84dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800c6f30 con 0x7fcd800d3710 2023-07-06T21:39:32.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd84dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd800c7210 con 0x7fcd800b83c0 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.476+0000 7fcd7dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 0x7fcd80150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 0x7fcd80150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38474/0 (socket says 172.21.15.84:38474) 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/2968031431 learned_addr learned my addr 172.21.15.84:0/2968031431 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7e59c700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd800b83c0 0x7fcd800c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7ed9d700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd800d3710 0x7fcd800cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd800b83c0 msgr2=0x7fcd800c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd800b83c0 0x7fcd800c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd800d3710 msgr2=0x7fcd800cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd800d3710 0x7fcd800cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 -- 172.21.15.84:0/2968031431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd740094a0 con 0x7fcd800cfa90 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7e59c700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd800b83c0 0x7fcd800c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd7dd9b700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 0x7fcd80150fa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009fd0 tx=0x7fcd74004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd740038a0 con 0x7fcd800cfa90 2023-07-06T21:39:32.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7401e070 con 0x7fcd800cfa90 2023-07-06T21:39:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74003c40 con 0x7fcd800cfa90 2023-07-06T21:39:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.477+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd800c74f0 con 0x7fcd800cfa90 2023-07-06T21:39:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.478+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd801514e0 con 0x7fcd800cfa90 2023-07-06T21:39:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.479+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd800c7a10 con 0x7fcd800cfa90 2023-07-06T21:39:32.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.479+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcd7402d450 con 0x7fcd800cfa90 2023-07-06T21:39:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.481+0000 7fcd6effd700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcd6006ace0 0x7fcd6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.481+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd7408a4e0 con 0x7fcd800cfa90 2023-07-06T21:39:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.481+0000 7fcd7e59c700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcd6006ace0 0x7fcd6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.481+0000 7fcd7e59c700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcd6006ace0 0x7fcd6006d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd70001270 tx=0x7fcd70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:32.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.485+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcd800c7a10 con 0x7fcd800cfa90 2023-07-06T21:39:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.682+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd800c7a10 con 0x7fcd800cfa90 2023-07-06T21:39:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.684+0000 7fcd6effd700 1 -- 172.21.15.84:0/2968031431 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7fcd800c7a10 con 0x7fcd800cfa90 2023-07-06T21:39:32.687 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:32.687 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcd6006ace0 msgr2=0x7fcd6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcd6006ace0 0x7fcd6006d190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcd70001270 tx=0x7fcd70009040 comp rx=0 tx=0).stop 2023-07-06T21:39:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 msgr2=0x7fcd80150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 0x7fcd80150fa0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcd74009fd0 tx=0x7fcd74004750 comp rx=0 tx=0).stop 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 shutdown_connections 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcd6006ace0 0x7fcd6006d190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd800d3710 0x7fcd800cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd800b83c0 0x7fcd800c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 --2- 172.21.15.84:0/2968031431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcd800cfa90 0x7fcd80150fa0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 >> 172.21.15.84:0/2968031431 conn(0x7fcd8006b350 msgr2=0x7fcd800b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 shutdown_connections 2023-07-06T21:39:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.686+0000 7fcd84dff700 1 -- 172.21.15.84:0/2968031431 wait complete. 2023-07-06T21:39:32.707 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-07-06T21:39:32.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:32.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.898+0000 7f181bbff700 1 Processor -- start 2023-07-06T21:39:32.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.898+0000 7f181bbff700 1 -- start start 2023-07-06T21:39:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.899+0000 7f181bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.899+0000 7f181bbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18140cfc60 0x7f18140d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.899+0000 7f181bbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18140d0a80 con 0x7f18140d3710 2023-07-06T21:39:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.899+0000 7f181bbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18140b8790 con 0x7f18140cfc60 2023-07-06T21:39:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.899+0000 7f181bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140b4e70 con 0x7f18140b83c0 2023-07-06T21:39:32.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18140cfc60 0x7f18140d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f18140cfc60 0x7f18140d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50680/0 (socket says 172.21.15.84:50680) 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 -- 172.21.15.84:0/3176801648 learned_addr learned my addr 172.21.15.84:0/3176801648 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181999b700 1 --2- 172.21.15.84:0/3176801648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 -- 172.21.15.84:0/3176801648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 msgr2=0x7f18140cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 --2- 172.21.15.84:0/3176801648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 -- 172.21.15.84:0/3176801648 >> v1:172.21.15.107:6789/0 conn(0x7f18140d3710 legacy=0x7f18140cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 -- 172.21.15.84:0/3176801648 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18140b5150 con 0x7f18140cfc60 2023-07-06T21:39:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.900+0000 7f181a19c700 1 --2- 172.21.15.84:0/3176801648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140cfc60 0x7f18140d0f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f180c009e50 tx=0x7f180c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c224eaf902ecae9c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f1818999700 1 -- 172.21.15.84:0/3176801648 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00e070 con 0x7f18140cfc60 2023-07-06T21:39:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f1818999700 1 -- 172.21.15.84:0/3176801648 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c00b9c0 con 0x7f18140cfc60 2023-07-06T21:39:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f1818999700 1 -- 172.21.15.84:0/3176801648 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00bd70 con 0x7f18140cfc60 2023-07-06T21:39:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 -- 172.21.15.84:0/3176801648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140cfc60 msgr2=0x7f18140d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 --2- 172.21.15.84:0/3176801648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140cfc60 0x7f18140d0f10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f180c009e50 tx=0x7f180c00b330 comp rx=0 tx=0).stop 2023-07-06T21:39:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 -- 172.21.15.84:0/3176801648 shutdown_connections 2023-07-06T21:39:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 --2- 172.21.15.84:0/3176801648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140cfc60 0x7f18140d0f10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 --2- 172.21.15.84:0/3176801648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 -- 172.21.15.84:0/3176801648 >> 172.21.15.84:0/3176801648 conn(0x7f181406b350 msgr2=0x7f181406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 -- 172.21.15.84:0/3176801648 shutdown_connections 2023-07-06T21:39:32.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.901+0000 7f181bbff700 1 -- 172.21.15.84:0/3176801648 wait complete. 2023-07-06T21:39:32.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.903+0000 7f181bbff700 1 Processor -- start 2023-07-06T21:39:32.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.903+0000 7f181bbff700 1 -- start start 2023-07-06T21:39:32.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181bbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181bbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18140cfc60 0x7f18140c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181bbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 0x7f1814153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181bbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140c7d70 con 0x7f18140d3710 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181bbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18140cb220 con 0x7f18140cfc60 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181bbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18140cb500 con 0x7f18140b83c0 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181a19c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 0x7f1814153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181a19c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 0x7f1814153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50686/0 (socket says 172.21.15.84:50686) 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181a19c700 1 -- 172.21.15.84:0/2904783268 learned_addr learned my addr 172.21.15.84:0/2904783268 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181999b700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.904+0000 7f181919a700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18140cfc60 0x7f18140c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181a19c700 1 -- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 msgr2=0x7f18140b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181a19c700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181a19c700 1 -- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18140cfc60 msgr2=0x7f18140c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:32.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181a19c700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18140cfc60 0x7f18140c72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181a19c700 1 -- 172.21.15.84:0/2904783268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009b30 con 0x7f18140d3710 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181a19c700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 0x7f1814153b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f180c012000 tx=0x7f180c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c0047c0 con 0x7f18140d3710 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c0099b0 con 0x7f18140d3710 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c01c680 con 0x7f18140d3710 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18140cb7e0 con 0x7f18140d3710 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.905+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18140b3950 con 0x7f18140d3710 2023-07-06T21:39:32.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.906+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18140cba20 con 0x7f18140d3710 2023-07-06T21:39:32.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.907+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f180c01c7e0 con 0x7f18140d3710 2023-07-06T21:39:32.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.908+0000 7f180a7fc700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17f806ace0 0x7f17f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:32.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.908+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f180c08ac60 con 0x7f18140d3710 2023-07-06T21:39:32.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.908+0000 7f181999b700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17f806ace0 0x7f17f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:32.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.909+0000 7f181999b700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17f806ace0 0x7f17f806d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1810007a30 tx=0x7f1810008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:32.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:32.912+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f18140cba20 con 0x7f18140d3710 2023-07-06T21:39:33.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.111+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18140cba20 con 0x7f18140d3710 2023-07-06T21:39:33.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.112+0000 7f180a7fc700 1 -- 172.21.15.84:0/2904783268 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f18140cba20 con 0x7f18140d3710 2023-07-06T21:39:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:33.116 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:33.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17f806ace0 msgr2=0x7f17f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17f806ace0 0x7f17f806d190 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1810007a30 tx=0x7f1810008040 comp rx=0 tx=0).stop 2023-07-06T21:39:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 msgr2=0x7f1814153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 0x7f1814153b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f180c012000 tx=0x7f180c004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 shutdown_connections 2023-07-06T21:39:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17f806ace0 0x7f17f806d190 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18140d3710 0x7f1814153b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f18140b83c0 0x7f18140b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 --2- 172.21.15.84:0/2904783268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18140cfc60 0x7f18140c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.114+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 >> 172.21.15.84:0/2904783268 conn(0x7f181406b350 msgr2=0x7f18140b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.115+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 shutdown_connections 2023-07-06T21:39:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.115+0000 7f181bbff700 1 -- 172.21.15.84:0/2904783268 wait complete. 2023-07-06T21:39:33.136 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-07-06T21:39:33.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-07-06T21:39:33.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-07-06T21:39:33.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-07-06T21:39:33.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-07-06T21:39:33.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-07-06T21:39:33.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-07-06T21:39:33.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-07-06T21:39:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.475+0000 7ff611dff700 1 Processor -- start 2023-07-06T21:39:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.476+0000 7ff611dff700 1 -- start start 2023-07-06T21:39:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.478+0000 7ff611dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff60c06a5a0 0x7ff60c158a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.478+0000 7ff611dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c159a60 con 0x7ff60c0adce0 2023-07-06T21:39:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.478+0000 7ff611dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c159780 con 0x7ff60c15ccc0 2023-07-06T21:39:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.478+0000 7ff611dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c15c1e0 con 0x7ff60c06a5a0 2023-07-06T21:39:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.479+0000 7ff60affd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff60c06a5a0 0x7ff60c158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.479+0000 7ff60affd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff60c06a5a0 0x7ff60c158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50702/0 (socket says 172.21.15.84:50702) 2023-07-06T21:39:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.479+0000 7ff60affd700 1 -- 172.21.15.84:0/21986449 learned_addr learned my addr 172.21.15.84:0/21986449 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.480+0000 7ff60affd700 1 -- 172.21.15.84:0/21986449 >> v1:172.21.15.107:6789/0 conn(0x7ff60c15ccc0 legacy=0x7ff60c158fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.480+0000 7ff60affd700 1 -- 172.21.15.84:0/21986449 >> v1:172.21.15.84:6789/0 conn(0x7ff60c0adce0 legacy=0x7ff60c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.480+0000 7ff60affd700 1 -- 172.21.15.84:0/21986449 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c15c4c0 con 0x7ff60c06a5a0 2023-07-06T21:39:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.481+0000 7ff60affd700 1 --2- 172.21.15.84:0/21986449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c06a5a0 0x7ff60c158a90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff60400d370 tx=0x7ff60400d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915598accfe109a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.482+0000 7ff60a7fc700 1 -- 172.21.15.84:0/21986449 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604010070 con 0x7ff60c06a5a0 2023-07-06T21:39:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.482+0000 7ff60a7fc700 1 -- 172.21.15.84:0/21986449 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60400dc90 con 0x7ff60c06a5a0 2023-07-06T21:39:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.482+0000 7ff60a7fc700 1 -- 172.21.15.84:0/21986449 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604007b30 con 0x7ff60c06a5a0 2023-07-06T21:39:33.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.483+0000 7ff611dff700 1 -- 172.21.15.84:0/21986449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c06a5a0 msgr2=0x7ff60c158a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.483+0000 7ff611dff700 1 --2- 172.21.15.84:0/21986449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c06a5a0 0x7ff60c158a90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff60400d370 tx=0x7ff60400d680 comp rx=0 tx=0).stop 2023-07-06T21:39:33.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.486+0000 7ff611dff700 1 -- 172.21.15.84:0/21986449 shutdown_connections 2023-07-06T21:39:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.486+0000 7ff611dff700 1 --2- 172.21.15.84:0/21986449 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c06a5a0 0x7ff60c158a90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.486+0000 7ff611dff700 1 -- 172.21.15.84:0/21986449 >> 172.21.15.84:0/21986449 conn(0x7ff60c0cb1c0 msgr2=0x7ff60c0cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.486+0000 7ff611dff700 1 -- 172.21.15.84:0/21986449 shutdown_connections 2023-07-06T21:39:33.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.487+0000 7ff611dff700 1 -- 172.21.15.84:0/21986449 wait complete. 2023-07-06T21:39:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.488+0000 7ff611dff700 1 Processor -- start 2023-07-06T21:39:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.489+0000 7ff611dff700 1 -- start start 2023-07-06T21:39:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff611dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff60c06a5a0 0x7ff60c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff611dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c0adce0 0x7ff60c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff611dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c15ccc0 0x7ff60c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff611dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c151830 con 0x7ff60c0adce0 2023-07-06T21:39:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff611dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c6b70 con 0x7ff60c15ccc0 2023-07-06T21:39:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff611dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0c6e50 con 0x7ff60c06a5a0 2023-07-06T21:39:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff60b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff60c06a5a0 0x7ff60c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff60b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff60c06a5a0 0x7ff60c1512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:51922/0 (socket says 172.21.15.84:51922) 2023-07-06T21:39:33.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff60b7fe700 1 -- 172.21.15.84:0/2271394486 learned_addr learned my addr 172.21.15.84:0/2271394486 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.490+0000 7ff60affd700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c0adce0 0x7ff60c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60bfff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c15ccc0 0x7ff60c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60affd700 1 -- 172.21.15.84:0/2271394486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff60c06a5a0 msgr2=0x7ff60c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60affd700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff60c06a5a0 0x7ff60c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60affd700 1 -- 172.21.15.84:0/2271394486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c15ccc0 msgr2=0x7ff60c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60affd700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff60c15ccc0 0x7ff60c15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60affd700 1 -- 172.21.15.84:0/2271394486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60400d050 con 0x7ff60c0adce0 2023-07-06T21:39:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60b7fe700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff60c06a5a0 0x7ff60c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.491+0000 7ff60affd700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c0adce0 0x7ff60c0c6160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff604017000 tx=0x7ff604004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7f6873dff700 1 Processor -- start 2023-07-06T21:39:33.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7f6873dff700 1 -- start start 2023-07-06T21:39:33.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7fe9565ff700 1 Processor -- start 2023-07-06T21:39:33.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604004550 con 0x7ff60c0adce0 2023-07-06T21:39:33.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6040046b0 con 0x7ff60c0adce0 2023-07-06T21:39:33.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604008030 con 0x7ff60c0adce0 2023-07-06T21:39:33.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c0c7130 con 0x7ff60c0adce0 2023-07-06T21:39:33.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.493+0000 7feca2bff700 1 Processor -- start 2023-07-06T21:39:33.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c0b1970 con 0x7ff60c0adce0 2023-07-06T21:39:33.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.492+0000 7fe9565ff700 1 -- start start 2023-07-06T21:39:33.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7fe9565ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe9500b83c0 0x7fe9500cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7fe9565ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9500d0a80 con 0x7fe9500d3710 2023-07-06T21:39:33.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7fe9565ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9500b8790 con 0x7fe9500cfc60 2023-07-06T21:39:33.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7fe9565ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9500b4e70 con 0x7fe9500b83c0 2023-07-06T21:39:33.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff5ec000fc0 con 0x7ff60c0adce0 2023-07-06T21:39:33.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7f6873dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0d3710 0x7f686c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7f6873dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c0cd910 con 0x7f686c0cfa90 2023-07-06T21:39:33.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7f6873dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f686c0cfee0 con 0x7f686c0b83c0 2023-07-06T21:39:33.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7f6873dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0d0a80 con 0x7f686c0d3710 2023-07-06T21:39:33.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7f6871b9b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f686c0b83c0 0x7f686c0cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:51748/0 (socket says 172.21.15.84:51748) 2023-07-06T21:39:33.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.494+0000 7f6871b9b700 1 -- 172.21.15.84:0/1878181567 learned_addr learned my addr 172.21.15.84:0/1878181567 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:33.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7fe954b9c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fe9500cfc60 0x7fe9500d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:51756/0 (socket says 172.21.15.84:51756) 2023-07-06T21:39:33.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7fe954b9c700 1 -- 172.21.15.84:0/3617331076 learned_addr learned my addr 172.21.15.84:0/3617331076 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:33.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.493+0000 7feca2bff700 1 -- start start 2023-07-06T21:39:33.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7feca2bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec9c0adce0 0x7fec9c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7feca2bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec9c1591a0 0x7fec9c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7feca2bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec9c159a60 con 0x7fec9c15ccc0 2023-07-06T21:39:33.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7feca2bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c159780 con 0x7fec9c1591a0 2023-07-06T21:39:33.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7feca2bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec9c15c1e0 con 0x7fec9c0adce0 2023-07-06T21:39:33.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254441762 0 0) 0x7f6864001490 con 0x7f686c0b83c0 2023-07-06T21:39:33.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68580035b0 con 0x7f686c0b83c0 2023-07-06T21:39:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591131392 0 0) 0x7f6868001040 con 0x7f686c0cfa90 2023-07-06T21:39:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6864001490 con 0x7f686c0cfa90 2023-07-06T21:39:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2636788930 0 0) 0x7f6864001490 con 0x7f686c0cfa90 2023-07-06T21:39:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0d3710 msgr2=0x7f686c0cf550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0d3710 0x7f686c0cf550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 >> v1:172.21.15.107:6789/0 conn(0x7f686c0b83c0 legacy=0x7f686c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0b8790 con 0x7f686c0cfa90 2023-07-06T21:39:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f68680016b0 con 0x7f686c0cfa90 2023-07-06T21:39:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 >> v1:172.21.15.84:6789/0 conn(0x7f686c0cfa90 legacy=0x7f686c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6858004c20 0x7f6858004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.495+0000 7fe94ffff700 1 --2- 172.21.15.84:0/3617331076 >> v2:172.21.15.107:3300/0 conn(0x7fe9500b83c0 0x7fe9500cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7fe94ffff700 1 -- 172.21.15.84:0/3617331076 >> v1:172.21.15.107:6789/0 conn(0x7fe9500cfc60 legacy=0x7fe9500d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7fe94ffff700 1 -- 172.21.15.84:0/3617331076 >> v1:172.21.15.84:6789/0 conn(0x7fe9500d3710 legacy=0x7fe9500cf550 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7fe94ffff700 1 -- 172.21.15.84:0/3617331076 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9500b5150 con 0x7fe9500b83c0 2023-07-06T21:39:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.496+0000 7fe94ffff700 1 --2- 172.21.15.84:0/3617331076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9500cd200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe938009840 tx=0x7fe938009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db5f83c64a0e1c64 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7fe94f7fe700 1 -- 172.21.15.84:0/3617331076 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938004230 con 0x7fe9500b83c0 2023-07-06T21:39:33.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7fe94f7fe700 1 -- 172.21.15.84:0/3617331076 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe938004390 con 0x7fe9500b83c0 2023-07-06T21:39:33.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7fe94f7fe700 1 -- 172.21.15.84:0/3617331076 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380036a0 con 0x7fe9500b83c0 2023-07-06T21:39:33.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7feca099b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec9c0adce0 0x7fec9c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68580037f0 0x7f6858003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68580043a0 0x7f685800ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0d0a80 con 0x7f68580037f0 2023-07-06T21:39:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685800c2b0 con 0x7f6858004c20 2023-07-06T21:39:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685800c470 con 0x7f68580043a0 2023-07-06T21:39:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7fec9bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fec9c15ccc0 0x7fec9c158a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47762/0 (socket says 172.21.15.84:47762) 2023-07-06T21:39:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7fec9bfff700 1 -- 172.21.15.84:0/1793117597 learned_addr learned my addr 172.21.15.84:0/1793117597 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.498+0000 7fe9565ff700 1 -- 172.21.15.84:0/3617331076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 msgr2=0x7fe9500cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.498+0000 7fe9565ff700 1 --2- 172.21.15.84:0/3617331076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9500cd200 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe938009840 tx=0x7fe938009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:33.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.498+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059041299 0 0) 0x7fec9c159a60 con 0x7fec9c15ccc0 2023-07-06T21:39:33.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.498+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec840035b0 con 0x7fec9c15ccc0 2023-07-06T21:39:33.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.497+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff604008190 con 0x7ff60c0adce0 2023-07-06T21:39:33.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.498+0000 7ff5f3fff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff5f806adb0 0x7ff5f806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.498+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff60408bd20 con 0x7ff60c0adce0 2023-07-06T21:39:33.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7ff5f3fff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7ff5f8073660 0x7ff5f8075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fe9565ff700 1 -- 172.21.15.84:0/3617331076 shutdown_connections 2023-07-06T21:39:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fe9565ff700 1 --2- 172.21.15.84:0/3617331076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9500cd200 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fe9565ff700 1 -- 172.21.15.84:0/3617331076 >> 172.21.15.84:0/3617331076 conn(0x7fe95006b350 msgr2=0x7fe95006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 --> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff5f8076120 con 0x7ff5f8073660 2023-07-06T21:39:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff60408c0d0 con 0x7ff60c0adce0 2023-07-06T21:39:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7f687239c700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68580043a0 0x7f685800ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7f6871b9b700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6858004c20 0x7f6858004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7ff60b7fe700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff5f806adb0 0x7ff5f806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7ff60bfff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7ff5f8073660 0x7ff5f8075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7fe9565ff700 1 -- 172.21.15.84:0/3617331076 shutdown_connections 2023-07-06T21:39:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3267054909 0 0) 0x7fec840035b0 con 0x7fec9c15ccc0 2023-07-06T21:39:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 >> v2:172.21.15.107:3300/0 conn(0x7fec9c0adce0 msgr2=0x7fec9c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 --2- 172.21.15.84:0/1793117597 >> v2:172.21.15.107:3300/0 conn(0x7fec9c0adce0 0x7fec9c069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 >> v2:172.21.15.84:3300/0 conn(0x7fec9c1591a0 msgr2=0x7fec9c06a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 --2- 172.21.15.84:0/1793117597 >> v2:172.21.15.84:3300/0 conn(0x7fec9c1591a0 0x7fec9c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c15c4c0 con 0x7fec9c15ccc0 2023-07-06T21:39:33.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fec90001670 con 0x7fec9c15ccc0 2023-07-06T21:39:33.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 >> v1:172.21.15.84:6789/0 conn(0x7fec9c15ccc0 legacy=0x7fec9c158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec840037f0 0x7fec84003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84004190 0x7fec8400b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8400b690 0x7fec8400db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c159780 con 0x7fec840037f0 2023-07-06T21:39:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.499+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c15c1e0 con 0x7fec84004190 2023-07-06T21:39:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8400e4f0 con 0x7fec8400b690 2023-07-06T21:39:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7f687239c700 1 -- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6858004c20 msgr2=0x7f6858004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7f687239c700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6858004c20 0x7f6858004ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7f687239c700 1 -- 172.21.15.84:0/1878181567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68580037f0 msgr2=0x7f6858003c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7f687239c700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68580037f0 0x7f6858003c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7f687239c700 1 -- 172.21.15.84:0/1878181567 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6864001490 con 0x7f68580043a0 2023-07-06T21:39:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7feca119c700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8400b690 0x7fec8400db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.501+0000 7fec9bfff700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84004190 0x7fec8400b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.501+0000 7f687239c700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68580043a0 0x7f685800ba90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6868007dd0 tx=0x7f6858005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b804bf409f401417 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.500+0000 7fe9565ff700 1 -- 172.21.15.84:0/3617331076 wait complete. 2023-07-06T21:39:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7feca099b700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec840037f0 0x7fec84003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7fec9bfff700 1 -- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8400b690 msgr2=0x7fec8400db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7fec9bfff700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8400b690 0x7fec8400db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7fec9bfff700 1 -- 172.21.15.84:0/1793117597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec840037f0 msgr2=0x7fec84003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7fec9bfff700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec840037f0 0x7fec84003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7fec9bfff700 1 -- 172.21.15.84:0/1793117597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c15c4c0 con 0x7fec84004190 2023-07-06T21:39:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.501+0000 7f3e901ff700 1 Processor -- start 2023-07-06T21:39:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7fe9565ff700 1 Processor -- start 2023-07-06T21:39:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.501+0000 7ff60b7fe700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff5f806adb0 0x7ff5f806d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff600001890 tx=0x7ff600009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.501+0000 7ff60bfff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7ff5f8073660 0x7ff5f8075a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7fec9bfff700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84004190 0x7fec8400b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c340 tx=0x7fec84004870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b8e4bb7e7e1bb2a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.501+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868007a60 con 0x7f68580043a0 2023-07-06T21:39:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686800cb30 con 0x7f68580043a0 2023-07-06T21:39:33.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7f6870b99700 1 -- 172.21.15.84:0/1878181567 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868012420 con 0x7f68580043a0 2023-07-06T21:39:33.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7f6873dff700 1 -- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68580043a0 msgr2=0x7f685800ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7f6873dff700 1 --2- 172.21.15.84:0/1878181567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68580043a0 0x7f685800ba90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6868007dd0 tx=0x7f6858005880 comp rx=0 tx=0).stop 2023-07-06T21:39:33.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.502+0000 7f687239c700 1 -- 172.21.15.84:0/1878181567 reap_dead start 2023-07-06T21:39:33.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7f6873dff700 1 -- 172.21.15.84:0/1878181567 shutdown_connections 2023-07-06T21:39:33.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7f6873dff700 1 -- 172.21.15.84:0/1878181567 >> 172.21.15.84:0/1878181567 conn(0x7f686c06b350 msgr2=0x7f686c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7f6873dff700 1 -- 172.21.15.84:0/1878181567 shutdown_connections 2023-07-06T21:39:33.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9000ce80 con 0x7fec84004190 2023-07-06T21:39:33.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec90005960 con 0x7fec84004190 2023-07-06T21:39:33.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7fec9b7fe700 1 -- 172.21.15.84:0/1793117597 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90005cf0 con 0x7fec84004190 2023-07-06T21:39:33.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7f6873dff700 1 -- 172.21.15.84:0/1878181567 wait complete. 2023-07-06T21:39:33.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7f3e901ff700 1 -- start start 2023-07-06T21:39:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7feca2bff700 1 -- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84004190 msgr2=0x7fec8400b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7feca2bff700 1 --2- 172.21.15.84:0/1793117597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec84004190 0x7fec8400b0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c340 tx=0x7fec84004870 comp rx=0 tx=0).stop 2023-07-06T21:39:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== osd.0 v2:172.21.15.84:6800/3639070743 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff5f8076120 con 0x7ff5f8073660 2023-07-06T21:39:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.503+0000 7fe9565ff700 1 -- start start 2023-07-06T21:39:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7fe9565ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9501f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7fe9565ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9500cfc60 0x7fe9500c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7fe9565ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9500d3710 0x7fe9500ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7fe9565ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501f1c60 con 0x7fe9500cfc60 2023-07-06T21:39:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7fe9565ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500cb8c0 con 0x7fe9500b83c0 2023-07-06T21:39:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.504+0000 7fe9565ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500c6bb0 con 0x7fe9500d3710 2023-07-06T21:39:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7feca119c700 1 -- 172.21.15.84:0/1793117597 reap_dead start 2023-07-06T21:39:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7feca2bff700 1 -- 172.21.15.84:0/1793117597 shutdown_connections 2023-07-06T21:39:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7feca2bff700 1 -- 172.21.15.84:0/1793117597 >> 172.21.15.84:0/1793117597 conn(0x7fec9c0cb1c0 msgr2=0x7fec9c0cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7fe94ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9501f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7fe94ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9501f1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58468/0 (socket says 172.21.15.84:58468) 2023-07-06T21:39:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7fe94ffff700 1 -- 172.21.15.84:0/214366898 learned_addr learned my addr 172.21.15.84:0/214366898 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7fe954b9c700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9500d3710 0x7fe9500ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7feca2bff700 1 -- 172.21.15.84:0/1793117597 shutdown_connections 2023-07-06T21:39:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7f6873dff700 1 Processor -- start 2023-07-06T21:39:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7f3e901ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3e8815ccc0 0x7f3e881591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7f3e901ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e88159a60 con 0x7f3e880adce0 2023-07-06T21:39:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7f3e901ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880ae120 con 0x7f3e880cbcd0 2023-07-06T21:39:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.505+0000 7f3e901ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e88159780 con 0x7f3e8815ccc0 2023-07-06T21:39:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7f3e8df9b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f3e880cbcd0 0x7f3e88158a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:51758/0 (socket says 172.21.15.84:51758) 2023-07-06T21:39:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7f3e8df9b700 1 -- 172.21.15.84:0/3724268419 learned_addr learned my addr 172.21.15.84:0/3724268419 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7f3e8d79a700 1 --2- 172.21.15.84:0/3724268419 >> v2:172.21.15.84:3300/0 conn(0x7f3e8815ccc0 0x7f3e881591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7fe9477fe700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9500cfc60 0x7fe9500c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7fe94ffff700 1 -- 172.21.15.84:0/214366898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9500d3710 msgr2=0x7fe9500ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7fe94ffff700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9500d3710 0x7fe9500ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7fe94ffff700 1 -- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9500cfc60 msgr2=0x7fe9500c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7fe94ffff700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9500cfc60 0x7fe9500c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7fe94ffff700 1 -- 172.21.15.84:0/214366898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9380094a0 con 0x7fe9500b83c0 2023-07-06T21:39:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7f6873dff700 1 -- start start 2023-07-06T21:39:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.506+0000 7feca2bff700 1 -- 172.21.15.84:0/1793117597 wait complete. 2023-07-06T21:39:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7fe94ffff700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9501f1720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe95006c6e0 tx=0x7fe9380047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2131843881 0 0) 0x7f3e880ae120 con 0x7f3e880cbcd0 2023-07-06T21:39:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e740035b0 con 0x7f3e880cbcd0 2023-07-06T21:39:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050273782 0 0) 0x7f3e88159a60 con 0x7f3e880adce0 2023-07-06T21:39:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e880ae120 con 0x7f3e880adce0 2023-07-06T21:39:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805809577 0 0) 0x7f3e880ae120 con 0x7f3e880adce0 2023-07-06T21:39:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 >> v2:172.21.15.84:3300/0 conn(0x7f3e8815ccc0 msgr2=0x7f3e881591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 --2- 172.21.15.84:0/3724268419 >> v2:172.21.15.84:3300/0 conn(0x7f3e8815ccc0 0x7f3e881591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 >> v1:172.21.15.107:6789/0 conn(0x7f3e880cbcd0 legacy=0x7f3e88158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8815c1e0 con 0x7f3e880adce0 2023-07-06T21:39:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f3e800019f0 con 0x7f3e880adce0 2023-07-06T21:39:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 >> v1:172.21.15.84:6789/0 conn(0x7f3e880adce0 legacy=0x7f3e880cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e74004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e740037f0 0x7f3e74003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e740041a0 0x7f3e7400dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88159780 con 0x7f3e740041a0 2023-07-06T21:39:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400e290 con 0x7f3e74004c20 2023-07-06T21:39:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400e450 con 0x7f3e740037f0 2023-07-06T21:39:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f6873dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68580043a0 0x7f686c1f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f6873dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c1f6f60 0x7f686c1f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f6873dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c1f9f60 0x7f686c1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f6873dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1f73d0 con 0x7f686c1f9f60 2023-07-06T21:39:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f6873dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1f6ad0 con 0x7f68580043a0 2023-07-06T21:39:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7f6873dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c1fa4e0 con 0x7f686c1f6f60 2023-07-06T21:39:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938011830 con 0x7fe9500b83c0 2023-07-06T21:39:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe938011990 con 0x7fe9500b83c0 2023-07-06T21:39:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.507+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938011d40 con 0x7fe9500b83c0 2023-07-06T21:39:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9500cb260 con 0x7fe9500b83c0 2023-07-06T21:39:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.508+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501f5870 con 0x7fe9500b83c0 2023-07-06T21:39:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.509+0000 7feca2bff700 1 Processor -- start 2023-07-06T21:39:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.509+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9500d0710 con 0x7fe9500b83c0 2023-07-06T21:39:33.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.509+0000 7f3e8d79a700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e740037f0 0x7f3e74003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.509+0000 7f3e8df9b700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e74004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.509+0000 7feca2bff700 1 -- start start 2023-07-06T21:39:33.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f687239c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c1f9f60 0x7f686c1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f687239c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c1f9f60 0x7f686c1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50762/0 (socket says 172.21.15.84:50762) 2023-07-06T21:39:33.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f687239c700 1 -- 172.21.15.84:0/752280045 learned_addr learned my addr 172.21.15.84:0/752280045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8e79c700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e740041a0 0x7f3e7400dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8df9b700 1 -- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e740037f0 msgr2=0x7f3e74003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8df9b700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e740037f0 0x7f3e74003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8df9b700 1 -- 172.21.15.84:0/3724268419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e740041a0 msgr2=0x7f3e7400dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8df9b700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e740041a0 0x7f3e7400dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8df9b700 1 -- 172.21.15.84:0/3724268419 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8815c1e0 con 0x7f3e74004c20 2023-07-06T21:39:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.510+0000 7f3e8e79c700 1 -- 172.21.15.84:0/3724268419 reap_dead start 2023-07-06T21:39:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f6871b9b700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68580043a0 0x7f686c1f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687239c700 1 -- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c1f6f60 msgr2=0x7f686c1f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687139a700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c1f6f60 0x7f686c1f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687239c700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c1f6f60 0x7f686c1f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687239c700 1 -- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68580043a0 msgr2=0x7f686c1f04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687239c700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68580043a0 0x7f686c1f04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687239c700 1 -- 172.21.15.84:0/752280045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c1fa7c0 con 0x7f686c1f9f60 2023-07-06T21:39:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f687139a700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c1f6f60 0x7f686c1f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f6871b9b700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68580043a0 0x7f686c1f04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7f3e8df9b700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e74004ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c001a20 tx=0x7f3e74005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b3369012bb327f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7feca2bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec9c1e69b0 0x7fec9c15a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7feca2bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06c3a0 0x7fec9c1e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7feca2bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec9c1e2e00 0x7fec9c1f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7feca2bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1e3270 con 0x7fec9c1e2e00 2023-07-06T21:39:33.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7feca2bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c06e1b0 con 0x7fec9c06c3a0 2023-07-06T21:39:33.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.511+0000 7feca2bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c06cb10 con 0x7fec9c1e69b0 2023-07-06T21:39:33.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec9c1e69b0 0x7fec9c15a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec9c1e69b0 0x7fec9c15a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:51978/0 (socket says 172.21.15.84:51978) 2023-07-06T21:39:33.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 -- 172.21.15.84:0/2894706565 learned_addr learned my addr 172.21.15.84:0/2894706565 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7fec9bfff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06c3a0 0x7fec9c1e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c004b30 con 0x7f3e74004c20 2023-07-06T21:39:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c004cb0 con 0x7f3e74004c20 2023-07-06T21:39:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7f3e8cf99700 1 -- 172.21.15.84:0/3724268419 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c0073b0 con 0x7f3e74004c20 2023-07-06T21:39:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe93801b440 con 0x7fe9500b83c0 2023-07-06T21:39:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 -- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06c3a0 msgr2=0x7fec9c1e28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec9c06c3a0 0x7fec9c1e28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 -- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec9c1e2e00 msgr2=0x7fec9c1f9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca119c700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec9c1e2e00 0x7fec9c1f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec9c1e2e00 0x7fec9c1f9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.512+0000 7feca099b700 1 -- 172.21.15.84:0/2894706565 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c15baa0 con 0x7fec9c1e69b0 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.513+0000 7feca119c700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec9c1e2e00 0x7fec9c1f9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.513+0000 7feca099b700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec9c1e69b0 0x7fec9c15a600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00e870 tx=0x7fec8c00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.513+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c015690 con 0x7fec9c1e69b0 2023-07-06T21:39:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.513+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c0157f0 con 0x7fec9c1e69b0 2023-07-06T21:39:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.513+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c015b80 con 0x7fec9c1e69b0 2023-07-06T21:39:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.514+0000 7fe94d7fa700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe93006ad30 0x7fe93006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.514+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe93808aef0 con 0x7fe9500b83c0 2023-07-06T21:39:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.514+0000 7fe94d7fa700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7fe9300735e0 0x7fe9300759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.515+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 --> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe9300760a0 con 0x7fe9300735e0 2023-07-06T21:39:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.515+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe93808b260 con 0x7fe9500b83c0 2023-07-06T21:39:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.513+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c15b4e0 con 0x7fec9c1e69b0 2023-07-06T21:39:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.514+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c06c810 con 0x7fec9c1e69b0 2023-07-06T21:39:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.515+0000 7fe954b9c700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7fe9300735e0 0x7fe9300759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.515+0000 7fe9477fe700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe93006ad30 0x7fe93006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.515+0000 7fcb6ddff700 1 Processor -- start 2023-07-06T21:39:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.516+0000 7fe954b9c700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7fe9300735e0 0x7fe9300759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.516+0000 7fe9477fe700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe93006ad30 0x7fe93006d1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9500d3b20 tx=0x7fe940008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.516+0000 7fcb6ddff700 1 -- start start 2023-07-06T21:39:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.517+0000 7f687239c700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c1f9f60 0x7f686c1594e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f685800c600 tx=0x7f6868001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.517+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868001670 con 0x7f686c1f9f60 2023-07-06T21:39:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.517+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686800ceb0 con 0x7f686c1f9f60 2023-07-06T21:39:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6868005270 con 0x7f686c1f9f60 2023-07-06T21:39:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c1f6760 con 0x7f686c1f9f60 2023-07-06T21:39:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c15e550 con 0x7f686c1f9f60 2023-07-06T21:39:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.517+0000 7f3e901ff700 1 -- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 msgr2=0x7f3e74004ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.517+0000 7f3e901ff700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e74004ff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c001a20 tx=0x7f3e74005880 comp rx=0 tx=0).stop 2023-07-06T21:39:33.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb6ddff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680b83c0 0x7fcb680cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb6ddff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb680d3710 0x7fcb680cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb6ddff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcb680cfa90 0x7fcb6815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb6ddff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680cfee0 con 0x7fcb680cfa90 2023-07-06T21:39:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb6ddff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680d0a80 con 0x7fcb680d3710 2023-07-06T21:39:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb6ddff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680b8790 con 0x7fcb680b83c0 2023-07-06T21:39:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb66ffd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb680d3710 0x7fcb680cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fcb67fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcb680cfa90 0x7fcb6815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb680d3710 0x7fcb680cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58502/0 (socket says 172.21.15.84:58502) 2023-07-06T21:39:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 -- 172.21.15.84:0/3512154384 learned_addr learned my addr 172.21.15.84:0/3512154384 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 -- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680b83c0 msgr2=0x7fcb680cd200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 --2- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680b83c0 0x7fcb680cd200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 -- 172.21.15.84:0/3512154384 >> v2:172.21.15.84:3300/0 conn(0x7fcb680cfa90 msgr2=0x7fcb6815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 --2- 172.21.15.84:0/3512154384 >> v2:172.21.15.84:3300/0 conn(0x7fcb680cfa90 0x7fcb6815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 -- 172.21.15.84:0/3512154384 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680b4e70 con 0x7fcb680d3710 2023-07-06T21:39:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.518+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec8c015ce0 con 0x7fec9c1e69b0 2023-07-06T21:39:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fec997fa700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec8806ace0 0x7fec8806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.517+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== osd.3 v2:172.21.15.84:6824/3293194179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe9300760a0 con 0x7fe9300735e0 2023-07-06T21:39:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.520+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec8c089ee0 con 0x7fec9c1e69b0 2023-07-06T21:39:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.519+0000 7fcb66ffd700 1 --2- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680d3710 0x7fcb680cf550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb6806c520 tx=0x7fcb6000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6705559e825c8d3c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.520+0000 7fcb667fc700 1 -- 172.21.15.84:0/3512154384 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000e070 con 0x7fcb680d3710 2023-07-06T21:39:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.520+0000 7fcb667fc700 1 -- 172.21.15.84:0/3512154384 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6000b910 con 0x7fcb680d3710 2023-07-06T21:39:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.520+0000 7fcb667fc700 1 -- 172.21.15.84:0/3512154384 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000bcc0 con 0x7fcb680d3710 2023-07-06T21:39:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.520+0000 7fec9bfff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec8806ace0 0x7fec8806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7f3e901ff700 1 -- 172.21.15.84:0/3724268419 shutdown_connections 2023-07-06T21:39:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7f3e901ff700 1 --2- 172.21.15.84:0/3724268419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e74004ff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7f3e901ff700 1 -- 172.21.15.84:0/3724268419 >> 172.21.15.84:0/3724268419 conn(0x7f3e880cb1c0 msgr2=0x7f3e880cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7f3e901ff700 1 -- 172.21.15.84:0/3724268419 shutdown_connections 2023-07-06T21:39:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7feca2bff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7fec9c0c94f0 0x7fec9c1fa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 --> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec9c15c7a0 con 0x7fec9c0c94f0 2023-07-06T21:39:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680d3710 msgr2=0x7fcb680cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680d3710 0x7fcb680cf550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcb6806c520 tx=0x7fcb6000b330 comp rx=0 tx=0).stop 2023-07-06T21:39:33.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fa0489ff700 1 Processor -- start 2023-07-06T21:39:33.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3512154384 shutdown_connections 2023-07-06T21:39:33.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3512154384 >> v2:172.21.15.84:3300/0 conn(0x7fcb680cfa90 0x7fcb6815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680b83c0 0x7fcb680cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3512154384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680d3710 0x7fcb680cf550 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3512154384 >> 172.21.15.84:0/3512154384 conn(0x7fcb6806b350 msgr2=0x7fcb6806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.522+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3512154384 shutdown_connections 2023-07-06T21:39:33.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.522+0000 7fa0489ff700 1 -- start start 2023-07-06T21:39:33.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7fa0489ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0440b2b80 0x7fa0440cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7fa0489ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 0x7fa04415b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.521+0000 7f3e901ff700 1 -- 172.21.15.84:0/3724268419 wait complete. 2023-07-06T21:39:33.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7f3e901ff700 1 Processor -- start 2023-07-06T21:39:33.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7fa0489ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0440ccc40 con 0x7fa0440b6370 2023-07-06T21:39:33.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7fa0489ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa04415b850 con 0x7fa0440b2b80 2023-07-06T21:39:33.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7fa0489ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440cb800 con 0x7fa0440cb040 2023-07-06T21:39:33.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.522+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3512154384 wait complete. 2023-07-06T21:39:33.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7fcb6ddff700 1 Processor -- start 2023-07-06T21:39:33.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.525+0000 7feca119c700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7fec9c0c94f0 0x7fec9c1fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.525+0000 7fcb6ddff700 1 -- start start 2023-07-06T21:39:33.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.523+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6868022070 con 0x7f686c1f9f60 2023-07-06T21:39:33.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.525+0000 7f68627fc700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f685006adb0 0x7f685006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.525+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f686808a300 con 0x7f686c1f9f60 2023-07-06T21:39:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.524+0000 7f3e901ff700 1 -- start start 2023-07-06T21:39:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e901ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e880c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e901ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e880c8770 0x7f3e8806d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e901ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e881515b0 0x7f3e8806d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e901ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b2560 con 0x7f3e880c8770 2023-07-06T21:39:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e901ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881fee20 con 0x7f3e74004c20 2023-07-06T21:39:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e901ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b1690 con 0x7f3e881515b0 2023-07-06T21:39:33.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.525+0000 7fa042d9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 0x7fa04415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa042d9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 0x7fa04415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:51994/0 (socket says 172.21.15.84:51994) 2023-07-06T21:39:33.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa042d9d700 1 -- 172.21.15.84:0/2298415130 learned_addr learned my addr 172.21.15.84:0/2298415130 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa042d9d700 1 -- 172.21.15.84:0/2298415130 >> v2:172.21.15.107:3300/0 conn(0x7fa0440b2b80 msgr2=0x7fa0440cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa042d9d700 1 --2- 172.21.15.84:0/2298415130 >> v2:172.21.15.107:3300/0 conn(0x7fa0440b2b80 0x7fa0440cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa042d9d700 1 -- 172.21.15.84:0/2298415130 >> v1:172.21.15.107:6789/0 conn(0x7fa0440b6370 legacy=0x7fa0440b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f6871b9b700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f685006adb0 0x7f685006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e8df9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e880c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e8df9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e880c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58526/0 (socket says 172.21.15.84:58526) 2023-07-06T21:39:33.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7f3e8df9b700 1 -- 172.21.15.84:0/3953406091 learned_addr learned my addr 172.21.15.84:0/3953406091 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa042d9d700 1 -- 172.21.15.84:0/2298415130 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440b3260 con 0x7fa0440cb040 2023-07-06T21:39:33.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.526+0000 7fa04159a700 1 -- 172.21.15.84:0/2298415130 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574924731 0 0) 0x7fa0440ccc40 con 0x7fa0440b6370 2023-07-06T21:39:33.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7f6873dff700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7f68540015e0 0x7f6854003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 --> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6854006bc0 con 0x7f68540015e0 2023-07-06T21:39:33.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7f687139a700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7f68540015e0 0x7f6854003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7f3e8e79c700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e881515b0 0x7f3e8806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7f3e8d79a700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e880c8770 0x7f3e8806d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7feca119c700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7fec9c0c94f0 0x7fec9c1fa520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fec9bfff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec8806ace0 0x7fec8806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec90000c00 tx=0x7fec90007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== osd.4 v2:172.21.15.107:6800/1667769091 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fec94009840 con 0x7fec9c0c94f0 2023-07-06T21:39:33.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fa042d9d700 1 --2- 172.21.15.84:0/2298415130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 0x7fa04415b310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa0340098f0 tx=0x7fa034009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f453005c99313cfd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fa04159a700 1 -- 172.21.15.84:0/2298415130 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03400b8a0 con 0x7fa0440cb040 2023-07-06T21:39:33.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fa04159a700 1 -- 172.21.15.84:0/2298415130 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03400ba00 con 0x7fa0440cb040 2023-07-06T21:39:33.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fa04159a700 1 -- 172.21.15.84:0/2298415130 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03400bd90 con 0x7fa0440cb040 2023-07-06T21:39:33.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb6ddff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680b83c0 0x7fcb681510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb6ddff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680cfa90 0x7fcb68159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb6ddff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb680d3710 0x7fcb681599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb6ddff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681f6440 con 0x7fcb680d3710 2023-07-06T21:39:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb6ddff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb681515e0 con 0x7fcb680b83c0 2023-07-06T21:39:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb6ddff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6815bdd0 con 0x7fcb680cfa90 2023-07-06T21:39:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb677fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680b83c0 0x7fcb681510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb677fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680b83c0 0x7fcb681510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58534/0 (socket says 172.21.15.84:58534) 2023-07-06T21:39:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb677fe700 1 -- 172.21.15.84:0/3980907670 learned_addr learned my addr 172.21.15.84:0/3980907670 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.527+0000 7fcb66ffd700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680cfa90 0x7fcb68159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fcb677fe700 1 -- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680cfa90 msgr2=0x7fcb68159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fcb677fe700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb680cfa90 0x7fcb68159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fcb677fe700 1 -- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb680d3710 msgr2=0x7fcb681599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fcb677fe700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb680d3710 0x7fcb681599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fcb677fe700 1 -- 172.21.15.84:0/3980907670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009b30 con 0x7fcb680b83c0 2023-07-06T21:39:33.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fa0489ff700 1 -- 172.21.15.84:0/2298415130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 msgr2=0x7fa04415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fa0489ff700 1 --2- 172.21.15.84:0/2298415130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 0x7fa04415b310 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa0340098f0 tx=0x7fa034009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:33.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7fcb677fe700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680b83c0 0x7fcb681510a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b470 tx=0x7fcb5000b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fa0489ff700 1 -- 172.21.15.84:0/2298415130 shutdown_connections 2023-07-06T21:39:33.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fa0489ff700 1 --2- 172.21.15.84:0/2298415130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440cb040 0x7fa04415b310 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fa0489ff700 1 --2- 172.21.15.84:0/2298415130 >> v2:172.21.15.107:3300/0 conn(0x7fa0440b2b80 0x7fa0440cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fa0489ff700 1 -- 172.21.15.84:0/2298415130 >> 172.21.15.84:0/2298415130 conn(0x7fa04406e640 msgr2=0x7fa04406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50011830 con 0x7fcb680b83c0 2023-07-06T21:39:33.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50011990 con 0x7fcb680b83c0 2023-07-06T21:39:33.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.528+0000 7f3e8d79a700 1 -- 172.21.15.84:0/3953406091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e881515b0 msgr2=0x7f3e8806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7f3e8d79a700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e881515b0 0x7f3e8806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7f3e8d79a700 1 -- 172.21.15.84:0/3953406091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 msgr2=0x7f3e880c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7f3e8d79a700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e74004c20 0x7f3e880c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50011d40 con 0x7fcb680b83c0 2023-07-06T21:39:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7f3e8d79a700 1 -- 172.21.15.84:0/3953406091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880b18e0 con 0x7f3e880c8770 2023-07-06T21:39:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fa0489ff700 1 -- 172.21.15.84:0/2298415130 shutdown_connections 2023-07-06T21:39:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7fa0489ff700 1 -- 172.21.15.84:0/2298415130 wait complete. 2023-07-06T21:39:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7f687139a700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7f68540015e0 0x7f6854003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.529+0000 7f3e8d79a700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e880c8770 0x7f3e8806d110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e88063bc0 tx=0x7f3e84012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.531+0000 7f6871b9b700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f685006adb0 0x7f685006d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6864001460 tx=0x7f6864009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.530+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8400c070 con 0x7f3e880c8770 2023-07-06T21:39:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.530+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84010c20 con 0x7f3e880c8770 2023-07-06T21:39:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.530+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84012e90 con 0x7f3e880c8770 2023-07-06T21:39:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.531+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6815c950 con 0x7fcb680b83c0 2023-07-06T21:39:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.531+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== osd.1 v2:172.21.15.84:6816/1216611025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6854006bc0 con 0x7f68540015e0 2023-07-06T21:39:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.531+0000 7fa0489ff700 1 Processor -- start 2023-07-06T21:39:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.531+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb680c8310 con 0x7fcb680b83c0 2023-07-06T21:39:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.531+0000 7fa0489ff700 1 -- start start 2023-07-06T21:39:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa0489ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0440b2b80 0x7fa044154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa0489ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440b6370 0x7fa0440c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa0489ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0440cb040 0x7fa0440d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa0489ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440c8cd0 con 0x7fa0440cb040 2023-07-06T21:39:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa0489ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440d18e0 con 0x7fa0440b2b80 2023-07-06T21:39:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa0489ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0440d1370 con 0x7fa0440b6370 2023-07-06T21:39:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa041d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440b6370 0x7fa0440c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa041d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440b6370 0x7fa0440c7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52016/0 (socket says 172.21.15.84:52016) 2023-07-06T21:39:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa041d9b700 1 -- 172.21.15.84:0/3237351687 learned_addr learned my addr 172.21.15.84:0/3237351687 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.533+0000 7fa04259c700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0440b2b80 0x7fa044154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa042d9d700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0440cb040 0x7fa0440d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa04259c700 1 -- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440b6370 msgr2=0x7fa0440c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa04259c700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0440b6370 0x7fa0440c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa04259c700 1 -- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0440cb040 msgr2=0x7fa0440d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa04259c700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0440cb040 0x7fa0440d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa04259c700 1 -- 172.21.15.84:0/3237351687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa034009550 con 0x7fa0440b2b80 2023-07-06T21:39:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa042d9d700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0440cb040 0x7fa0440d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e880b1bc0 con 0x7f3e880c8770 2023-07-06T21:39:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e880b2180 con 0x7f3e880c8770 2023-07-06T21:39:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fa04259c700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0440b2b80 0x7fa044154b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c430 tx=0x7fa03800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.535+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380054e0 con 0x7fa0440b2b80 2023-07-06T21:39:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.535+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa038003680 con 0x7fa0440b2b80 2023-07-06T21:39:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.535+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa038003a30 con 0x7fa0440b2b80 2023-07-06T21:39:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.535+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0440c8150 con 0x7fa0440b2b80 2023-07-06T21:39:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.535+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0440c8710 con 0x7fa0440b2b80 2023-07-06T21:39:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.536+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa0440ccea0 con 0x7fa0440b2b80 2023-07-06T21:39:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.534+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb5001a400 con 0x7fcb680b83c0 2023-07-06T21:39:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.536+0000 7fcb5ffff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb4806adb0 0x7fcb4806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.536+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb500894a0 con 0x7fcb680b83c0 2023-07-06T21:39:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.537+0000 7fcb66ffd700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb4806adb0 0x7fcb4806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.538+0000 7fcb66ffd700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb4806adb0 0x7fcb4806d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb680d3b20 tx=0x7fcb60009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.538+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa038003b90 con 0x7fa0440b2b80 2023-07-06T21:39:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.539+0000 7fa032ffd700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa02406adb0 0x7fa02406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.539+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa038088fe0 con 0x7fa0440b2b80 2023-07-06T21:39:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.540+0000 7fa032ffd700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fa024073660 0x7fa024075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.540+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 --> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa024076120 con 0x7fa024073660 2023-07-06T21:39:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.540+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa038089390 con 0x7fa0440b2b80 2023-07-06T21:39:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.540+0000 7fa042d9d700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fa024073660 0x7fa024075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.541+0000 7fa041d9b700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa02406adb0 0x7fa02406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.542+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e8815c7a0 con 0x7f3e880c8770 2023-07-06T21:39:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.542+0000 7fa041d9b700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa02406adb0 0x7fa02406d260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa044159ef0 tx=0x7fa03c0095b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.542+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7fcb680b25d0 0x7fcb6815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.542+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 --> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb680b5150 con 0x7fcb680b25d0 2023-07-06T21:39:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.542+0000 7fa042d9d700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fa024073660 0x7fa024075a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.543+0000 7fcb67fff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7fcb680b25d0 0x7fcb6815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.543+0000 7fcb67fff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7fcb680b25d0 0x7fcb6815cfd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.544+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== osd.5 v2:172.21.15.107:6808/3858042795 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa024076120 con 0x7fa024073660 2023-07-06T21:39:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.542+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3e84013440 con 0x7f3e880c8770 2023-07-06T21:39:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.544+0000 7f3e7a7fc700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3e6c06adb0 0x7f3e6c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.544+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3e84089cc0 con 0x7f3e880c8770 2023-07-06T21:39:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.544+0000 7f3e7a7fc700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7f3e6c0735e0 0x7f3e6c0759f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.544+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 --> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e6c0760a0 con 0x7f3e6c0735e0 2023-07-06T21:39:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.544+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== osd.6 v2:172.21.15.107:6816/2301301903 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcb580096c0 con 0x7fcb680b25d0 2023-07-06T21:39:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.545+0000 7f3e8e79c700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7f3e6c0735e0 0x7f3e6c0759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.545+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e84054d50 con 0x7f3e880c8770 2023-07-06T21:39:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.546+0000 7fe082dff700 1 Processor -- start 2023-07-06T21:39:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.546+0000 7f3e8e79c700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7f3e6c0735e0 0x7f3e6c0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.547+0000 7fe082dff700 1 -- start start 2023-07-06T21:39:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.549+0000 7fe082dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe07c0adce0 0x7fe07c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.549+0000 7fe082dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe07c15ccc0 0x7fe07c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.549+0000 7fe082dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07c159a60 con 0x7fe07c158a90 2023-07-06T21:39:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.549+0000 7fe082dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07c159780 con 0x7fe07c0adce0 2023-07-06T21:39:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.549+0000 7fe082dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07c15c1e0 con 0x7fe07c15ccc0 2023-07-06T21:39:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.550+0000 7fe080b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe07c0adce0 0x7fe07c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.550+0000 7fe080b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe07c0adce0 0x7fe07c069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50826/0 (socket says 172.21.15.84:50826) 2023-07-06T21:39:33.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.550+0000 7fe080b9b700 1 -- 172.21.15.84:0/2796748491 learned_addr learned my addr 172.21.15.84:0/2796748491 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.550+0000 7fe07bfff700 1 --2- 172.21.15.84:0/2796748491 >> v2:172.21.15.107:3300/0 conn(0x7fe07c15ccc0 0x7fe07c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.550+0000 7fe080b9b700 1 -- 172.21.15.84:0/2796748491 >> v2:172.21.15.107:3300/0 conn(0x7fe07c15ccc0 msgr2=0x7fe07c06a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.550+0000 7fe080b9b700 1 --2- 172.21.15.84:0/2796748491 >> v2:172.21.15.107:3300/0 conn(0x7fe07c15ccc0 0x7fe07c06a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.551+0000 7fe080b9b700 1 -- 172.21.15.84:0/2796748491 >> v1:172.21.15.84:6789/0 conn(0x7fe07c158a90 legacy=0x7fe07c158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.551+0000 7fe080b9b700 1 -- 172.21.15.84:0/2796748491 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c15c4c0 con 0x7fe07c0adce0 2023-07-06T21:39:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.551+0000 7fe080b9b700 1 --2- 172.21.15.84:0/2796748491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 0x7fe07c069e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0092e0 tx=0x7fe06c0095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738b7ae94638c2c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.552+0000 7fe07b7fe700 1 -- 172.21.15.84:0/2796748491 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c004230 con 0x7fe07c0adce0 2023-07-06T21:39:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.552+0000 7fe07b7fe700 1 -- 172.21.15.84:0/2796748491 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c004390 con 0x7fe07c0adce0 2023-07-06T21:39:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.552+0000 7fe07b7fe700 1 -- 172.21.15.84:0/2796748491 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0036a0 con 0x7fe07c0adce0 2023-07-06T21:39:33.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.552+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== osd.2 v2:172.21.15.84:6808/3137136770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3e6c0760a0 con 0x7f3e6c0735e0 2023-07-06T21:39:33.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.552+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 --> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff5ec002d40 con 0x7ff5f8073660 2023-07-06T21:39:33.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.549+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 --> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9500b5430 con 0x7fe9300735e0 2023-07-06T21:39:33.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.551+0000 7fe94d7fa700 1 -- 172.21.15.84:0/214366898 <== osd.3 v2:172.21.15.84:6824/3293194179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe948001ec0 con 0x7fe9300735e0 2023-07-06T21:39:33.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7fe082dff700 1 -- 172.21.15.84:0/2796748491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 msgr2=0x7fe07c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7fe082dff700 1 --2- 172.21.15.84:0/2796748491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 0x7fe07c069e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe06c0092e0 tx=0x7fe06c0095f0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7ff5f3fff700 1 -- 172.21.15.84:0/2271394486 <== osd.0 v2:172.21.15.84:6800/3639070743 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff5ec002d40 con 0x7ff5f8073660 2023-07-06T21:39:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7fe082dff700 1 -- 172.21.15.84:0/2796748491 shutdown_connections 2023-07-06T21:39:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7fe082dff700 1 --2- 172.21.15.84:0/2796748491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 0x7fe07c069e90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7fe082dff700 1 --2- 172.21.15.84:0/2796748491 >> v2:172.21.15.107:3300/0 conn(0x7fe07c15ccc0 0x7fe07c06a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.553+0000 7fe082dff700 1 -- 172.21.15.84:0/2796748491 >> 172.21.15.84:0/2796748491 conn(0x7fe07c0cb1c0 msgr2=0x7fe07c0cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.554+0000 7fe082dff700 1 -- 172.21.15.84:0/2796748491 shutdown_connections 2023-07-06T21:39:33.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.554+0000 7f3e8df9b700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3e6c06adb0 0x7f3e6c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.554+0000 7fe082dff700 1 -- 172.21.15.84:0/2796748491 wait complete. 2023-07-06T21:39:33.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.555+0000 7f3e8df9b700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3e6c06adb0 0x7f3e6c06d260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005fd0 tx=0x7f3e7c00a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.556+0000 7fe082dff700 1 Processor -- start 2023-07-06T21:39:33.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.556+0000 7fe082dff700 1 -- start start 2023-07-06T21:39:33.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.557+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7ff5f8073660 msgr2=0x7ff5f8075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.557+0000 7ff611dff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7ff5f8073660 0x7ff5f8075a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.557+0000 7fe082dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 0x7fe07c14d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe082dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe07c158a90 0x7fe07c0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe082dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c15ccc0 0x7fe07c06d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe082dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c0b4560 con 0x7fe07c0adce0 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe082dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c14d8a0 con 0x7fe07c15ccc0 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe082dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c0b38e0 con 0x7fe07c158a90 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe07c158a90 0x7fe07c0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe080b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 0x7fe07c14d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe07bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe07c158a90 0x7fe07c0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52032/0 (socket says 172.21.15.84:52032) 2023-07-06T21:39:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe07bfff700 1 -- 172.21.15.84:0/1327374038 learned_addr learned my addr 172.21.15.84:0/1327374038 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.558+0000 7fe08139c700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c15ccc0 0x7fe07c06d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe07bfff700 1 -- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c15ccc0 msgr2=0x7fe07c06d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe07bfff700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c15ccc0 0x7fe07c06d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe07bfff700 1 -- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 msgr2=0x7fe07c14d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe07bfff700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe07c0adce0 0x7fe07c14d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe07bfff700 1 -- 172.21.15.84:0/1327374038 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c009de0 con 0x7fe07c158a90 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe08139c700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe07c15ccc0 0x7fe07c06d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe07bfff700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe07c158a90 0x7fe07c0b3020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe07400efc0 tx=0x7fe074009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.560+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400cae0 con 0x7fe07c158a90 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.560+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07400f040 con 0x7fe07c158a90 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.560+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c0b3bc0 con 0x7fe07c158a90 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.560+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400ce60 con 0x7fe07c158a90 2023-07-06T21:39:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.560+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c0b4180 con 0x7fe07c158a90 2023-07-06T21:39:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.561+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe07c0ae0b0 con 0x7fe07c158a90 2023-07-06T21:39:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.562+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe074014480 con 0x7fe07c158a90 2023-07-06T21:39:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.563+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 --> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec9c0cbcd0 con 0x7fec9c0c94f0 2023-07-06T21:39:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fec997fa700 1 -- 172.21.15.84:0/2894706565 <== osd.4 v2:172.21.15.107:6800/1667769091 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec94009b20 con 0x7fec9c0c94f0 2023-07-06T21:39:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fe0797fa700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe06806ace0 0x7fe06806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe07408a950 con 0x7fe07c158a90 2023-07-06T21:39:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fe0797fa700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fe068073510 0x7fe068075920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 --> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe068075fd0 con 0x7fe068073510 2023-07-06T21:39:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe07408acc0 con 0x7fe07c158a90 2023-07-06T21:39:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.564+0000 7fe080b9b700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe06806ace0 0x7fe06806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7fe08139c700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fe068073510 0x7fe068075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7fec9c0c94f0 msgr2=0x7fec9c1fa520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca2bff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7fec9c0c94f0 0x7fec9c1fa520 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7fe080b9b700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe06806ace0 0x7fe06806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe06c000c00 tx=0x7fe06c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec8806ace0 msgr2=0x7fec8806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca2bff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec8806ace0 0x7fec8806d190 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec90000c00 tx=0x7fec90007000 comp rx=0 tx=0).stop 2023-07-06T21:39:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec9c1e69b0 msgr2=0x7fec9c15a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca2bff700 1 --2- 172.21.15.84:0/2894706565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec9c1e69b0 0x7fec9c15a600 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fec8c00e870 tx=0x7fec8c00eb80 comp rx=0 tx=0).stop 2023-07-06T21:39:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7feca119c700 1 -- 172.21.15.84:0/2894706565 reap_dead start 2023-07-06T21:39:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.565+0000 7fe08139c700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fe068073510 0x7fe068075920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff5f806adb0 msgr2=0x7ff5f806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff5f806adb0 0x7ff5f806d260 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff600001890 tx=0x7ff600009040 comp rx=0 tx=0).stop 2023-07-06T21:39:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c0adce0 msgr2=0x7ff60c0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 --2- 172.21.15.84:0/2271394486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff60c0adce0 0x7ff60c0c6160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff604017000 tx=0x7ff604004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff60bfff700 1 -- 172.21.15.84:0/2271394486 reap_dead start 2023-07-06T21:39:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 shutdown_connections 2023-07-06T21:39:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 >> 172.21.15.84:0/2271394486 conn(0x7ff60c0cb1c0 msgr2=0x7ff60c0cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7fe9300735e0 msgr2=0x7fe9300759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.559+0000 7fe9565ff700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7fe9300735e0 0x7fe9300759f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.562+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe93006ad30 msgr2=0x7fe93006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 shutdown_connections 2023-07-06T21:39:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 >> 172.21.15.84:0/2894706565 conn(0x7fec9c0cb1c0 msgr2=0x7fec9c070650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.562+0000 7fe9565ff700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe93006ad30 0x7fe93006d1e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe9500d3b20 tx=0x7fe940008040 comp rx=0 tx=0).stop 2023-07-06T21:39:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.562+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 msgr2=0x7fe9501f1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.562+0000 7fe9565ff700 1 --2- 172.21.15.84:0/214366898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9500b83c0 0x7fe9501f1720 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe95006c6e0 tx=0x7fe9380047c0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.562+0000 7fe954b9c700 1 -- 172.21.15.84:0/214366898 reap_dead start 2023-07-06T21:39:33.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.563+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 shutdown_connections 2023-07-06T21:39:33.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.563+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 >> 172.21.15.84:0/214366898 conn(0x7fe95006b350 msgr2=0x7fe9500d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.563+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 shutdown_connections 2023-07-06T21:39:33.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.563+0000 7fe9565ff700 1 -- 172.21.15.84:0/214366898 wait complete. 2023-07-06T21:39:33.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 shutdown_connections 2023-07-06T21:39:33.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.567+0000 7ff611dff700 1 -- 172.21.15.84:0/2271394486 wait complete. 2023-07-06T21:39:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 shutdown_connections 2023-07-06T21:39:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.567+0000 7feca2bff700 1 -- 172.21.15.84:0/2894706565 wait complete. 2023-07-06T21:39:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.566+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 --> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6854005ca0 con 0x7f68540015e0 2023-07-06T21:39:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.569+0000 7f68627fc700 1 -- 172.21.15.84:0/752280045 <== osd.1 v2:172.21.15.84:6816/1216611025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6854005ca0 con 0x7f68540015e0 2023-07-06T21:39:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.570+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 --> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb680d0710 con 0x7fcb680b25d0 2023-07-06T21:39:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.571+0000 7fcb5ffff700 1 -- 172.21.15.84:0/3980907670 <== osd.6 v2:172.21.15.107:6816/2301301903 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb58005a20 con 0x7fcb680b25d0 2023-07-06T21:39:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.571+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7fcb680b25d0 msgr2=0x7fcb6815cfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.571+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7fcb680b25d0 0x7fcb6815cfd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb4806adb0 msgr2=0x7fcb4806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb4806adb0 0x7fcb4806d260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb680d3b20 tx=0x7fcb60009000 comp rx=0 tx=0).stop 2023-07-06T21:39:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680b83c0 msgr2=0x7fcb681510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 --2- 172.21.15.84:0/3980907670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb680b83c0 0x7fcb681510a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb5000b470 tx=0x7fcb5000b830 comp rx=0 tx=0).stop 2023-07-06T21:39:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb67fff700 1 -- 172.21.15.84:0/3980907670 reap_dead start 2023-07-06T21:39:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 shutdown_connections 2023-07-06T21:39:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 >> 172.21.15.84:0/3980907670 conn(0x7fcb6806b350 msgr2=0x7fcb680d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== osd.7 v2:172.21.15.107:6824/440998536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe068075fd0 con 0x7fe068073510 2023-07-06T21:39:33.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.574+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 shutdown_connections 2023-07-06T21:39:33.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.575+0000 7fcb6ddff700 1 -- 172.21.15.84:0/3980907670 wait complete. 2023-07-06T21:39:33.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.579+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7f68540015e0 msgr2=0x7f6854003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.579+0000 7f6873dff700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7f68540015e0 0x7f6854003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.578+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 --> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa0440cb520 con 0x7fa024073660 2023-07-06T21:39:33.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.580+0000 7fa032ffd700 1 -- 172.21.15.84:0/3237351687 <== osd.5 v2:172.21.15.107:6808/3858042795 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa03400bb70 con 0x7fa024073660 2023-07-06T21:39:33.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.584+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fa024073660 msgr2=0x7fa024075a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.584+0000 7fa0489ff700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fa024073660 0x7fa024075a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa02406adb0 msgr2=0x7fa02406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa02406adb0 0x7fa02406d260 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa044159ef0 tx=0x7fa03c0095b0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0440b2b80 msgr2=0x7fa044154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 --2- 172.21.15.84:0/3237351687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0440b2b80 0x7fa044154b10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa03800c430 tx=0x7fa03800c7f0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa042d9d700 1 -- 172.21.15.84:0/3237351687 reap_dead start 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 shutdown_connections 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 >> 172.21.15.84:0/3237351687 conn(0x7fa04406e640 msgr2=0x7fa0440b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 shutdown_connections 2023-07-06T21:39:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.585+0000 7fa0489ff700 1 -- 172.21.15.84:0/3237351687 wait complete. 2023-07-06T21:39:33.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.586+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f685006adb0 msgr2=0x7f685006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.586+0000 7f6873dff700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f685006adb0 0x7f685006d260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6864001460 tx=0x7f6864009040 comp rx=0 tx=0).stop 2023-07-06T21:39:33.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.586+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c1f9f60 msgr2=0x7f686c1594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.586+0000 7f6873dff700 1 --2- 172.21.15.84:0/752280045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c1f9f60 0x7f686c1594e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f685800c600 tx=0x7f6868001970 comp rx=0 tx=0).stop 2023-07-06T21:39:33.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.586+0000 7f687239c700 1 -- 172.21.15.84:0/752280045 reap_dead start 2023-07-06T21:39:33.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.587+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 shutdown_connections 2023-07-06T21:39:33.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.587+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 >> 172.21.15.84:0/752280045 conn(0x7f686c06b350 msgr2=0x7f686c0c5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.587+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 --> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3e8815c4c0 con 0x7f3e6c0735e0 2023-07-06T21:39:33.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.588+0000 7f3e7a7fc700 1 -- 172.21.15.84:0/3953406091 <== osd.2 v2:172.21.15.84:6808/3137136770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3e80001910 con 0x7f3e6c0735e0 2023-07-06T21:39:33.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.588+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 shutdown_connections 2023-07-06T21:39:33.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.588+0000 7f6873dff700 1 -- 172.21.15.84:0/752280045 wait complete. 2023-07-06T21:39:33.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.590+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7f3e6c0735e0 msgr2=0x7f3e6c0759f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.590+0000 7f3e901ff700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7f3e6c0735e0 0x7f3e6c0759f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3e6c06adb0 msgr2=0x7f3e6c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e901ff700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3e6c06adb0 0x7f3e6c06d260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c005fd0 tx=0x7f3e7c00a000 comp rx=0 tx=0).stop 2023-07-06T21:39:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e880c8770 msgr2=0x7f3e8806d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e901ff700 1 --2- 172.21.15.84:0/3953406091 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3e880c8770 0x7f3e8806d110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f3e88063bc0 tx=0x7f3e84012c50 comp rx=0 tx=0).stop 2023-07-06T21:39:33.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e8e79c700 1 -- 172.21.15.84:0/3953406091 reap_dead start 2023-07-06T21:39:33.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 shutdown_connections 2023-07-06T21:39:33.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.591+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 >> 172.21.15.84:0/3953406091 conn(0x7f3e880cb1c0 msgr2=0x7f3e880afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.592+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 shutdown_connections 2023-07-06T21:39:33.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.592+0000 7f3e901ff700 1 -- 172.21.15.84:0/3953406091 wait complete. 2023-07-06T21:39:33.677 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:33.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T21:39:33.678 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-07-06T21:39:33.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.598+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 --> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe07c15c7a0 con 0x7fe068073510 2023-07-06T21:39:33.679 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T21:39:33.680 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:33.681 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T21:39:33.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.599+0000 7fe0797fa700 1 -- 172.21.15.84:0/1327374038 <== osd.7 v2:172.21.15.107:6824/440998536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe070001850 con 0x7fe068073510 2023-07-06T21:39:33.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fe068073510 msgr2=0x7fe068075920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fe068073510 0x7fe068075920 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:33.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe06806ace0 msgr2=0x7fe06806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe06806ace0 0x7fe06806d190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe06c000c00 tx=0x7fe06c008030 comp rx=0 tx=0).stop 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe07c158a90 msgr2=0x7fe07c0b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 --2- 172.21.15.84:0/1327374038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe07c158a90 0x7fe07c0b3020 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe07400efc0 tx=0x7fe074009f80 comp rx=0 tx=0).stop 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe08139c700 1 -- 172.21.15.84:0/1327374038 reap_dead start 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 shutdown_connections 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 >> 172.21.15.84:0/1327374038 conn(0x7fe07c0cb1c0 msgr2=0x7fe07c158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 shutdown_connections 2023-07-06T21:39:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:33.600+0000 7fe082dff700 1 -- 172.21.15.84:0/1327374038 wait complete. 2023-07-06T21:39:33.684 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:33.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T21:39:33.686 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:33.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T21:39:33.687 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-07-06T21:39:33.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T21:39:33.688 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:33.688 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T21:39:33.689 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-07-06T21:39:33.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T21:39:34.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.064+0000 7f9205dff700 1 Processor -- start 2023-07-06T21:39:34.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.064+0000 7f9205dff700 1 -- start start 2023-07-06T21:39:34.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.066+0000 7f9205dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92000c9ed0 0x7f92000ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.066+0000 7f9205dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9200069e90 0x7f920006a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.066+0000 7f9205dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92000c91c0 con 0x7f920015c4b0 2023-07-06T21:39:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.066+0000 7f9205dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92000ca860 con 0x7f9200069e90 2023-07-06T21:39:34.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.066+0000 7f9205dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920006a810 con 0x7f92000c9ed0 2023-07-06T21:39:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.067+0000 7f91ff7fe700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f920015c4b0 0x7f92000c3a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:51778/0 (socket says 172.21.15.84:51778) 2023-07-06T21:39:34.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.067+0000 7f91ff7fe700 1 -- 172.21.15.84:0/403487384 learned_addr learned my addr 172.21.15.84:0/403487384 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.066+0000 7efec57ff700 1 Processor -- start 2023-07-06T21:39:34.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.067+0000 7efec57ff700 1 -- start start 2023-07-06T21:39:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.069+0000 7efec57ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efec01608e0 0x7efec0069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.069+0000 7efec57ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec0071050 con 0x7efec00c4dd0 2023-07-06T21:39:34.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.069+0000 7efec57ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec0164890 con 0x7efec00c52f0 2023-07-06T21:39:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.069+0000 7efec57ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec00c91c0 con 0x7efec01608e0 2023-07-06T21:39:34.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.069+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783967359 0 0) 0x7f92000c91c0 con 0x7f920015c4b0 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.069+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91e40035b0 con 0x7f920015c4b0 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fffff700 1 --2- 172.21.15.84:0/403487384 >> v2:172.21.15.107:3300/0 conn(0x7f9200069e90 0x7f920006a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017749246 0 0) 0x7f91e40035b0 con 0x7f920015c4b0 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92000c9ed0 msgr2=0x7f92000ca320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92000c9ed0 0x7f92000ca320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 >> v2:172.21.15.107:3300/0 conn(0x7f9200069e90 msgr2=0x7f920006a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 --2- 172.21.15.84:0/403487384 >> v2:172.21.15.107:3300/0 conn(0x7f9200069e90 0x7f920006a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000cd350 con 0x7f920015c4b0 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f91f80019f0 con 0x7f920015c4b0 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 >> v1:172.21.15.107:6789/0 conn(0x7f920015c4b0 legacy=0x7f92000c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91e4003a00 0x7f91e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4004850 0x7f91e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e4005200 0x7f91e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92000ca860 con 0x7f91e4004850 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920006a810 con 0x7f91e4005200 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e400c1e0 con 0x7f91e4003a00 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.070+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91e4003a00 0x7f91e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91fffff700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e4005200 0x7f91e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91ff7fe700 1 -- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e4005200 msgr2=0x7f91e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e4005200 0x7f91e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91ff7fe700 1 -- 172.21.15.84:0/403487384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4004850 msgr2=0x7f91e4004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e4004850 0x7f91e4004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91ff7fe700 1 -- 172.21.15.84:0/403487384 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92000cd350 con 0x7f91e4003a00 2023-07-06T21:39:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.071+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91e4003a00 0x7f91e4003e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f91f800a8c0 tx=0x7f91e4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aecc43dc8dc833ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.072+0000 7efebf7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7efec00c4dd0 0x7efec0070940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47784/0 (socket says 172.21.15.84:47784) 2023-07-06T21:39:34.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.072+0000 7efebf7fe700 1 -- 172.21.15.84:0/2562592784 learned_addr learned my addr 172.21.15.84:0/2562592784 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.072+0000 7efebeffd700 1 --2- 172.21.15.84:0/2562592784 >> v2:172.21.15.107:3300/0 conn(0x7efec01608e0 0x7efec0069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.072+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8012600 con 0x7f91e4003a00 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.072+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f8012780 con 0x7f91e4003a00 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.072+0000 7f91fe7fc700 1 -- 172.21.15.84:0/403487384 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8012b30 con 0x7f91e4003a00 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7f9205dff700 1 -- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91e4003a00 msgr2=0x7f91e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7f9205dff700 1 --2- 172.21.15.84:0/403487384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91e4003a00 0x7f91e4003e10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f91f800a8c0 tx=0x7f91e4005880 comp rx=0 tx=0).stop 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7f91fffff700 1 -- 172.21.15.84:0/403487384 reap_dead start 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7f9205dff700 1 -- 172.21.15.84:0/403487384 shutdown_connections 2023-07-06T21:39:34.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7f9205dff700 1 -- 172.21.15.84:0/403487384 >> 172.21.15.84:0/403487384 conn(0x7f92000cb1c0 msgr2=0x7f92000cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7f9205dff700 1 -- 172.21.15.84:0/403487384 shutdown_connections 2023-07-06T21:39:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.074+0000 7f9205dff700 1 -- 172.21.15.84:0/403487384 wait complete. 2023-07-06T21:39:34.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924343226 0 0) 0x7efec0071050 con 0x7efec00c4dd0 2023-07-06T21:39:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.073+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea80035b0 con 0x7efec00c4dd0 2023-07-06T21:39:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.074+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621170938 0 0) 0x7efec0164890 con 0x7efec00c52f0 2023-07-06T21:39:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.074+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efec0071050 con 0x7efec00c52f0 2023-07-06T21:39:34.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.074+0000 7efebeffd700 1 -- 172.21.15.84:0/2562592784 >> v1:172.21.15.107:6789/0 conn(0x7efec00c52f0 legacy=0x7efec0070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.074+0000 7efebeffd700 1 -- 172.21.15.84:0/2562592784 >> v1:172.21.15.84:6789/0 conn(0x7efec00c4dd0 legacy=0x7efec0070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.074+0000 7efebeffd700 1 -- 172.21.15.84:0/2562592784 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec0164b70 con 0x7efec01608e0 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.075+0000 7efebeffd700 1 --2- 172.21.15.84:0/2562592784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec01608e0 0x7efec0069e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009840 tx=0x7efeb0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17fb094ee4547816 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.076+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0011830 con 0x7efec01608e0 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.076+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0011990 con 0x7efec01608e0 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.076+0000 7efebdffb700 1 -- 172.21.15.84:0/2562592784 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0011d20 con 0x7efec01608e0 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.077+0000 7efec57ff700 1 -- 172.21.15.84:0/2562592784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec01608e0 msgr2=0x7efec0069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.077+0000 7efec57ff700 1 --2- 172.21.15.84:0/2562592784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec01608e0 0x7efec0069e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009840 tx=0x7efeb0009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.076+0000 7f9205dff700 1 Processor -- start 2023-07-06T21:39:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.077+0000 7efec57ff700 1 -- 172.21.15.84:0/2562592784 shutdown_connections 2023-07-06T21:39:34.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.077+0000 7efec57ff700 1 --2- 172.21.15.84:0/2562592784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec01608e0 0x7efec0069e90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.077+0000 7efec57ff700 1 -- 172.21.15.84:0/2562592784 >> 172.21.15.84:0/2562592784 conn(0x7efec00cb1c0 msgr2=0x7efec00cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.078+0000 7efec57ff700 1 -- 172.21.15.84:0/2562592784 shutdown_connections 2023-07-06T21:39:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.077+0000 7f9205dff700 1 -- start start 2023-07-06T21:39:34.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f9205dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001fa140 0x7f920015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f9205dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92001fa510 0x7f92001f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f9205dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 0x7f92001f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f9205dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920015f950 con 0x7f92001fa510 2023-07-06T21:39:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f9205dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920015ed40 con 0x7f92001fa140 2023-07-06T21:39:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f9205dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920015f020 con 0x7f92001f3b10 2023-07-06T21:39:34.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f550bdff700 1 Processor -- start 2023-07-06T21:39:34.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f550bdff700 1 -- start start 2023-07-06T21:39:34.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.078+0000 7efec57ff700 1 -- 172.21.15.84:0/2562592784 wait complete. 2023-07-06T21:39:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f91fffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 0x7f92001f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f91fffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 0x7f92001f3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52048/0 (socket says 172.21.15.84:52048) 2023-07-06T21:39:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.079+0000 7f91fffff700 1 -- 172.21.15.84:0/2647628027 learned_addr learned my addr 172.21.15.84:0/2647628027 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.080+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001fa140 0x7f920015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.080+0000 7f91fffff700 1 -- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001fa140 msgr2=0x7f920015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.080+0000 7f91fffff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001fa140 0x7f920015e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.080+0000 7f91fffff700 1 -- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92001fa510 msgr2=0x7f92001f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.080+0000 7f91fffff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92001fa510 0x7f92001f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.080+0000 7f91fffff700 1 -- 172.21.15.84:0/2647628027 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920015f300 con 0x7f92001f3b10 2023-07-06T21:39:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.081+0000 7efec57ff700 1 Processor -- start 2023-07-06T21:39:34.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.082+0000 7f550bdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f55040c91c0 0x7f5504070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.082+0000 7f550bdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5504069e90 0x7f550406a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.082+0000 7f550bdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55040ae160 con 0x7f55040add90 2023-07-06T21:39:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.082+0000 7f550bdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55040c9660 con 0x7f55040c91c0 2023-07-06T21:39:34.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.082+0000 7f550bdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550416cae0 con 0x7f5504069e90 2023-07-06T21:39:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.082+0000 7f91fffff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 0x7f92001f3f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f920006b760 tx=0x7f91ec0117e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00c070 con 0x7f92001f3b10 2023-07-06T21:39:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ec015870 con 0x7f92001f3b10 2023-07-06T21:39:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec015c00 con 0x7f92001f3b10 2023-07-06T21:39:34.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920015f5e0 con 0x7f92001f3b10 2023-07-06T21:39:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5504069e90 0x7f550406a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5504069e90 0x7f550406a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58612/0 (socket says 172.21.15.84:58612) 2023-07-06T21:39:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 -- 172.21.15.84:0/3923458396 learned_addr learned my addr 172.21.15.84:0/3923458396 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92001fa980 con 0x7f92001f3b10 2023-07-06T21:39:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.081+0000 7efec57ff700 1 -- start start 2023-07-06T21:39:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7efec57ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec00c4dd0 0x7efec0201e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7efec57ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 0x7efec006dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7efec57ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efec01608e0 0x7efec006e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7efec57ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01ff3d0 con 0x7efec01608e0 2023-07-06T21:39:34.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7efec57ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec01ff6b0 con 0x7efec00c52f0 2023-07-06T21:39:34.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7efec57ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec01ff990 con 0x7efec00c4dd0 2023-07-06T21:39:34.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 -- 172.21.15.84:0/3923458396 >> v2:172.21.15.84:3300/0 conn(0x7f55040c91c0 msgr2=0x7f5504070c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 --2- 172.21.15.84:0/3923458396 >> v2:172.21.15.84:3300/0 conn(0x7f55040c91c0 0x7f5504070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 -- 172.21.15.84:0/3923458396 >> v1:172.21.15.107:6789/0 conn(0x7f55040add90 legacy=0x7f55040ccc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.083+0000 7f550a39c700 1 -- 172.21.15.84:0/3923458396 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5504071180 con 0x7f5504069e90 2023-07-06T21:39:34.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7efebe7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 0x7efec006dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7f550a39c700 1 --2- 172.21.15.84:0/3923458396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5504069e90 0x7f550406a2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f550416dd70 tx=0x7f54fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8957c4969cfa2f6d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7efebf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efec01608e0 0x7efec006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7efebe7fc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 0x7efec006dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58628/0 (socket says 172.21.15.84:58628) 2023-07-06T21:39:34.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7efebe7fc700 1 -- 172.21.15.84:0/3597798026 learned_addr learned my addr 172.21.15.84:0/3597798026 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7f5508b99700 1 -- 172.21.15.84:0/3923458396 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00e070 con 0x7f5504069e90 2023-07-06T21:39:34.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7f5508b99700 1 -- 172.21.15.84:0/3923458396 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc003a20 con 0x7f5504069e90 2023-07-06T21:39:34.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7f5508b99700 1 -- 172.21.15.84:0/3923458396 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc003dd0 con 0x7f5504069e90 2023-07-06T21:39:34.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.084+0000 7fb8703ff700 1 Processor -- start 2023-07-06T21:39:34.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebeffd700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec00c4dd0 0x7efec0201e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebe7fc700 1 -- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec00c4dd0 msgr2=0x7efec0201e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebe7fc700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec00c4dd0 0x7efec0201e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebe7fc700 1 -- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efec01608e0 msgr2=0x7efec006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebe7fc700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efec01608e0 0x7efec006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebe7fc700 1 -- 172.21.15.84:0/3597798026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00094a0 con 0x7efec00c52f0 2023-07-06T21:39:34.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7ff6971ff700 1 Processor -- start 2023-07-06T21:39:34.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7efebf7fe700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efec01608e0 0x7efec006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:34.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7efebe7fc700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 0x7efec006dc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c4f0 tx=0x7efeb400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7fb8703ff700 1 -- start start 2023-07-06T21:39:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.085+0000 7f550bdff700 1 -- 172.21.15.84:0/3923458396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5504069e90 msgr2=0x7f550406a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7f550bdff700 1 --2- 172.21.15.84:0/3923458396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5504069e90 0x7f550406a2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f550416dd70 tx=0x7f54fc00b330 comp rx=0 tx=0).stop 2023-07-06T21:39:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7f550bdff700 1 -- 172.21.15.84:0/3923458396 shutdown_connections 2023-07-06T21:39:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7f550bdff700 1 --2- 172.21.15.84:0/3923458396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5504069e90 0x7f550406a2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7f550bdff700 1 --2- 172.21.15.84:0/3923458396 >> v2:172.21.15.84:3300/0 conn(0x7f55040c91c0 0x7f5504070c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7f550bdff700 1 -- 172.21.15.84:0/3923458396 >> 172.21.15.84:0/3923458396 conn(0x7f55040cb1c0 msgr2=0x7f55040cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb40052d0 con 0x7efec00c52f0 2023-07-06T21:39:34.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb4005430 con 0x7efec00c52f0 2023-07-06T21:39:34.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4003880 con 0x7efec00c52f0 2023-07-06T21:39:34.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91ec0143e0 con 0x7f92001f3b10 2023-07-06T21:39:34.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92000c67d0 con 0x7f92001f3b10 2023-07-06T21:39:34.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7ff6971ff700 1 -- start start 2023-07-06T21:39:34.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.086+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec01ffc70 con 0x7efec00c52f0 2023-07-06T21:39:34.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec01ff080 con 0x7efec00c52f0 2023-07-06T21:39:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7f91f7fff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91e806ade0 0x7f91e806d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91ec0892f0 con 0x7f92001f3b10 2023-07-06T21:39:34.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7f550bdff700 1 -- 172.21.15.84:0/3923458396 shutdown_connections 2023-07-06T21:39:34.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7ff6971ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff6900d3710 0x7ff6900cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7ff6971ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff6900cfa90 0x7ff6900d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7ff6971ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6900cd910 con 0x7ff6900b83c0 2023-07-06T21:39:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7ff6971ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900cff10 con 0x7ff6900d3710 2023-07-06T21:39:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7ff6971ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6900d0a80 con 0x7ff6900cfa90 2023-07-06T21:39:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7ff694f9b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7ff6900b83c0 0x7ff6900cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:51806/0 (socket says 172.21.15.84:51806) 2023-07-06T21:39:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7ff694f9b700 1 -- 172.21.15.84:0/2750188067 learned_addr learned my addr 172.21.15.84:0/2750188067 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7ff69579c700 1 --2- 172.21.15.84:0/2750188067 >> v2:172.21.15.107:3300/0 conn(0x7ff6900cfa90 0x7ff6900d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7fb8703ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86806a5a0 0x7fb868158a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7fb8703ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb86815ccc0 0x7fb868158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7fb8703ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb868159a60 con 0x7fb8680cbcd0 2023-07-06T21:39:34.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7fb8703ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb868159510 con 0x7fb86815ccc0 2023-07-06T21:39:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.087+0000 7fb8703ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86815c1e0 con 0x7fb86806a5a0 2023-07-06T21:39:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7ff69579c700 1 -- 172.21.15.84:0/2750188067 >> v2:172.21.15.84:3300/0 conn(0x7ff6900d3710 msgr2=0x7ff6900cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7ff69579c700 1 --2- 172.21.15.84:0/2750188067 >> v2:172.21.15.84:3300/0 conn(0x7ff6900d3710 0x7ff6900cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7ff69579c700 1 -- 172.21.15.84:0/2750188067 >> v1:172.21.15.107:6789/0 conn(0x7ff6900b83c0 legacy=0x7ff6900cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7ff69579c700 1 -- 172.21.15.84:0/2750188067 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6900b8790 con 0x7ff6900cfa90 2023-07-06T21:39:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91e806ade0 0x7f91e806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7fb86d99a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86806a5a0 0x7fb868158a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7fb86d99a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86806a5a0 0x7fb868158a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52060/0 (socket says 172.21.15.84:52060) 2023-07-06T21:39:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7fb86d99a700 1 -- 172.21.15.84:0/2393252068 learned_addr learned my addr 172.21.15.84:0/2393252068 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7fb86e99c700 1 --2- 172.21.15.84:0/2393252068 >> v2:172.21.15.84:3300/0 conn(0x7fb86815ccc0 0x7fb868158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.089+0000 7ff69579c700 1 --2- 172.21.15.84:0/2750188067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900cfa90 0x7ff6900d0f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6900b1b00 tx=0x7ff68800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7504b8d93f6094 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7f91ff7fe700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91e806ade0 0x7f91e806d290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f920006a950 tx=0x7f91f8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7fb86e99c700 1 -- 172.21.15.84:0/2393252068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86806a5a0 msgr2=0x7fb868158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7fb86e99c700 1 --2- 172.21.15.84:0/2393252068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86806a5a0 0x7fb868158a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7fb86e99c700 1 -- 172.21.15.84:0/2393252068 >> v1:172.21.15.84:6789/0 conn(0x7fb8680cbcd0 legacy=0x7fb868069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7fb86e99c700 1 -- 172.21.15.84:0/2393252068 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86815c4c0 con 0x7fb86815ccc0 2023-07-06T21:39:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7fb86e99c700 1 --2- 172.21.15.84:0/2393252068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb86815ccc0 0x7fb868158fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb86000f370 tx=0x7fb86000f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40975b0c1f91290 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.088+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec00c37e0 con 0x7efec00c52f0 2023-07-06T21:39:34.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.090+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efeb40039e0 con 0x7efec00c52f0 2023-07-06T21:39:34.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7fb86d199700 1 -- 172.21.15.84:0/2393252068 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860012070 con 0x7fb86815ccc0 2023-07-06T21:39:34.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7fb86d199700 1 -- 172.21.15.84:0/2393252068 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86000fe30 con 0x7fb86815ccc0 2023-07-06T21:39:34.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7fb86d199700 1 -- 172.21.15.84:0/2393252068 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860003c60 con 0x7fb86815ccc0 2023-07-06T21:39:34.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7f550bdff700 1 -- 172.21.15.84:0/3923458396 wait complete. 2023-07-06T21:39:34.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7ff68f7fe700 1 -- 172.21.15.84:0/2750188067 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68800e070 con 0x7ff6900cfa90 2023-07-06T21:39:34.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7ff68f7fe700 1 -- 172.21.15.84:0/2750188067 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68800bad0 con 0x7ff6900cfa90 2023-07-06T21:39:34.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7ff68f7fe700 1 -- 172.21.15.84:0/2750188067 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68800be60 con 0x7ff6900cfa90 2023-07-06T21:39:34.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7ff6971ff700 1 -- 172.21.15.84:0/2750188067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900cfa90 msgr2=0x7ff6900d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7ff6971ff700 1 --2- 172.21.15.84:0/2750188067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900cfa90 0x7ff6900d0f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff6900b1b00 tx=0x7ff68800b330 comp rx=0 tx=0).stop 2023-07-06T21:39:34.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.091+0000 7efea77fe700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efeac06ad30 0x7efeac06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efeb40887c0 con 0x7efec00c52f0 2023-07-06T21:39:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7fb8703ff700 1 -- 172.21.15.84:0/2393252068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb86815ccc0 msgr2=0x7fb868158fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2393252068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb86815ccc0 0x7fb868158fd0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb86000f370 tx=0x7fb86000f680 comp rx=0 tx=0).stop 2023-07-06T21:39:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7fb8703ff700 1 -- 172.21.15.84:0/2393252068 shutdown_connections 2023-07-06T21:39:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2393252068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb86815ccc0 0x7fb868158fd0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2393252068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86806a5a0 0x7fb868158a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.092+0000 7fb8703ff700 1 -- 172.21.15.84:0/2393252068 >> 172.21.15.84:0/2393252068 conn(0x7fb8680cb1c0 msgr2=0x7fb8680cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7efebeffd700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efeac06ad30 0x7efeac06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7ff6971ff700 1 -- 172.21.15.84:0/2750188067 shutdown_connections 2023-07-06T21:39:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7ff6971ff700 1 --2- 172.21.15.84:0/2750188067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900cfa90 0x7ff6900d0f10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7ff6971ff700 1 --2- 172.21.15.84:0/2750188067 >> v2:172.21.15.84:3300/0 conn(0x7ff6900d3710 0x7ff6900cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7ff6971ff700 1 -- 172.21.15.84:0/2750188067 >> 172.21.15.84:0/2750188067 conn(0x7ff69006b350 msgr2=0x7ff69006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7fb8703ff700 1 -- 172.21.15.84:0/2393252068 shutdown_connections 2023-07-06T21:39:34.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7f550bdff700 1 Processor -- start 2023-07-06T21:39:34.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.094+0000 7f550bdff700 1 -- start start 2023-07-06T21:39:34.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.093+0000 7fb8703ff700 1 -- 172.21.15.84:0/2393252068 wait complete. 2023-07-06T21:39:34.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.094+0000 7efebeffd700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efeac06ad30 0x7efeac06d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.094+0000 7ff6971ff700 1 -- 172.21.15.84:0/2750188067 shutdown_connections 2023-07-06T21:39:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550bdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5504069e90 0x7f55040c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550bdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 0x7f55040c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550bdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55040c91c0 0x7f55040af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550bdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550415a960 con 0x7f55040add90 2023-07-06T21:39:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550bdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550415b750 con 0x7f55040c91c0 2023-07-06T21:39:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550bdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55040c6d60 con 0x7f5504069e90 2023-07-06T21:39:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.094+0000 7ff6971ff700 1 -- 172.21.15.84:0/2750188067 wait complete. 2023-07-06T21:39:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f5509b9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5504069e90 0x7f55040c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550939a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 0x7f55040c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550939a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 0x7f55040c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50890/0 (socket says 172.21.15.84:50890) 2023-07-06T21:39:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f550939a700 1 -- 172.21.15.84:0/3019291093 learned_addr learned my addr 172.21.15.84:0/3019291093 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.095+0000 7f5509b9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5504069e90 0x7f55040c6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52072/0 (socket says 172.21.15.84:52072) 2023-07-06T21:39:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7fb8703ff700 1 Processor -- start 2023-07-06T21:39:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7f550939a700 1 -- 172.21.15.84:0/3019291093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5504069e90 msgr2=0x7f55040c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7f550939a700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5504069e90 0x7f55040c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7f550939a700 1 -- 172.21.15.84:0/3019291093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55040c91c0 msgr2=0x7f55040af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7f550939a700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55040c91c0 0x7f55040af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7f550939a700 1 -- 172.21.15.84:0/3019291093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc009b30 con 0x7f55040add90 2023-07-06T21:39:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7f550939a700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 0x7f55040c7bc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54f400b4a0 tx=0x7f54f400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.096+0000 7fb8703ff700 1 -- start start 2023-07-06T21:39:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7ff6971ff700 1 Processor -- start 2023-07-06T21:39:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4011830 con 0x7f55040add90 2023-07-06T21:39:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f4011990 con 0x7f55040add90 2023-07-06T21:39:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4011d20 con 0x7f55040add90 2023-07-06T21:39:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550415ae20 con 0x7f55040add90 2023-07-06T21:39:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7ff6971ff700 1 -- start start 2023-07-06T21:39:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb8703ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb86806a5a0 0x7fb8680b2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb8703ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8680cbcd0 0x7fb8680b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb8703ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 0x7fb8681514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb8703ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86806d3f0 con 0x7fb8680cbcd0 2023-07-06T21:39:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb8703ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86806d690 con 0x7fb86806a5a0 2023-07-06T21:39:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb8703ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb86806d970 con 0x7fb86815ccc0 2023-07-06T21:39:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.097+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550415b3e0 con 0x7f55040add90 2023-07-06T21:39:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7ff6971ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6900b83c0 0x7ff6900b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7ff6971ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6900cfa90 0x7ff690071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb86e99c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 0x7fb8681514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb86e99c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 0x7fb8681514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52086/0 (socket says 172.21.15.84:52086) 2023-07-06T21:39:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.098+0000 7fb86e99c700 1 -- 172.21.15.84:0/2913265708 learned_addr learned my addr 172.21.15.84:0/2913265708 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff6971ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900d3710 0x7ff6900ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff6971ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015f8b0 con 0x7ff6900b83c0 2023-07-06T21:39:34.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff2cbfff700 1 Processor -- start 2023-07-06T21:39:34.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff6971ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015e810 con 0x7ff6900d3710 2023-07-06T21:39:34.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff6971ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69015eaf0 con 0x7ff6900cfa90 2023-07-06T21:39:34.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff68ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6900cfa90 0x7ff690071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff69579c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900d3710 0x7ff6900ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff68ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6900cfa90 0x7ff690071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52092/0 (socket says 172.21.15.84:52092) 2023-07-06T21:39:34.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff68ffff700 1 -- 172.21.15.84:0/323866487 learned_addr learned my addr 172.21.15.84:0/323866487 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7fb86e99c700 1 -- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb86806a5a0 msgr2=0x7fb8680b2160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-06T21:39:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7fb86e99c700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb86806a5a0 0x7fb8680b2160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7fb86e99c700 1 -- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8680cbcd0 msgr2=0x7fb8680b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7fb86e99c700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8680cbcd0 0x7fb8680b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7fb86e99c700 1 -- 172.21.15.84:0/2913265708 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86000f050 con 0x7fb86815ccc0 2023-07-06T21:39:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.099+0000 7ff2cbfff700 1 -- start start 2023-07-06T21:39:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7ff69579c700 1 -- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6900cfa90 msgr2=0x7ff690071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7ff69579c700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6900cfa90 0x7ff690071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7ff69579c700 1 -- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6900b83c0 msgr2=0x7ff6900b7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7ff69579c700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6900b83c0 0x7ff6900b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7ff69579c700 1 -- 172.21.15.84:0/323866487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff688009a70 con 0x7ff6900d3710 2023-07-06T21:39:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7fb86e99c700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 0x7fb8681514f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb860004000 tx=0x7fb860004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.100+0000 7ff69579c700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900d3710 0x7ff6900ca6c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6900cf520 tx=0x7ff688004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec0052f0 con 0x7f55040add90 2023-07-06T21:39:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.102+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92000c67d0 con 0x7f92001f3b10 2023-07-06T21:39:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 0x7ff2c41591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7ff2cbfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c4159a60 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7ff2cbfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c4159780 con 0x7ff2c406a5a0 2023-07-06T21:39:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c415c1e0 con 0x7ff2c415ccc0 2023-07-06T21:39:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.102+0000 7ff2ca59c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 0x7ff2c41591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.102+0000 7ff2ca59c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 0x7ff2c41591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52094/0 (socket says 172.21.15.84:52094) 2023-07-06T21:39:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.102+0000 7ff2ca59c700 1 -- 172.21.15.84:0/493279003 learned_addr learned my addr 172.21.15.84:0/493279003 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000b940 con 0x7fb86815ccc0 2023-07-06T21:39:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86000baa0 con 0x7fb86815ccc0 2023-07-06T21:39:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000be50 con 0x7fb86815ccc0 2023-07-06T21:39:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86806dc50 con 0x7fb86815ccc0 2023-07-06T21:39:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86806e210 con 0x7fb86815ccc0 2023-07-06T21:39:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.101+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54f401a400 con 0x7f55040add90 2023-07-06T21:39:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7ff2ca59c700 1 -- 172.21.15.84:0/493279003 >> v1:172.21.15.107:6789/0 conn(0x7ff2c406a5a0 legacy=0x7ff2c4158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7ff2ca59c700 1 -- 172.21.15.84:0/493279003 >> v1:172.21.15.84:6789/0 conn(0x7ff2c40cbcd0 legacy=0x7ff2c4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7ff2ca59c700 1 -- 172.21.15.84:0/493279003 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c415c4c0 con 0x7ff2c415ccc0 2023-07-06T21:39:34.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efeb4053900 con 0x7efec00c52f0 2023-07-06T21:39:34.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb868155860 con 0x7fb86815ccc0 2023-07-06T21:39:34.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7f54fa7fc700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54e806ad30 0x7f54e806d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f54f4089310 con 0x7f55040add90 2023-07-06T21:39:34.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.103+0000 7ff2ca59c700 1 --2- 172.21.15.84:0/493279003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 0x7ff2c41591a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009e80 tx=0x7ff2bc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c983342fe3a50511 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.104+0000 7f5509b9b700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54e806ad30 0x7f54e806d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7f5509b9b700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54e806ad30 0x7f54e806d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55040c9f90 tx=0x7f5500008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.104+0000 7ff2c8d99700 1 -- 172.21.15.84:0/493279003 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc00e070 con 0x7ff2c415ccc0 2023-07-06T21:39:34.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.104+0000 7ff2c8d99700 1 -- 172.21.15.84:0/493279003 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2bc003a20 con 0x7ff2c415ccc0 2023-07-06T21:39:34.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.104+0000 7ff2c8d99700 1 -- 172.21.15.84:0/493279003 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc003dd0 con 0x7ff2c415ccc0 2023-07-06T21:39:34.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7ff2cbfff700 1 -- 172.21.15.84:0/493279003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 msgr2=0x7ff2c41591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/493279003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 0x7ff2c41591a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc009e80 tx=0x7ff2bc00b330 comp rx=0 tx=0).stop 2023-07-06T21:39:34.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7ff2cbfff700 1 -- 172.21.15.84:0/493279003 shutdown_connections 2023-07-06T21:39:34.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/493279003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c415ccc0 0x7ff2c41591a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7ff2cbfff700 1 -- 172.21.15.84:0/493279003 >> 172.21.15.84:0/493279003 conn(0x7ff2c40cb1c0 msgr2=0x7ff2c40cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.105+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb860009070 con 0x7fb86815ccc0 2023-07-06T21:39:34.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.106+0000 7ff2cbfff700 1 -- 172.21.15.84:0/493279003 shutdown_connections 2023-07-06T21:39:34.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.106+0000 7fb85effd700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb84c06ad30 0x7fb84c06d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.107+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb86008ab40 con 0x7fb86815ccc0 2023-07-06T21:39:34.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.108+0000 7fcb879ff700 1 Processor -- start 2023-07-06T21:39:34.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.108+0000 7fcb879ff700 1 -- start start 2023-07-06T21:39:34.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.107+0000 7ff2cbfff700 1 -- 172.21.15.84:0/493279003 wait complete. 2023-07-06T21:39:34.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.109+0000 7ff2cbfff700 1 Processor -- start 2023-07-06T21:39:34.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.109+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb860055e30 con 0x7fb86815ccc0 2023-07-06T21:39:34.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.109+0000 7ff2cbfff700 1 -- start start 2023-07-06T21:39:34.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.111+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2c406a5a0 0x7ff2c406d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.111+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 0x7ff2c40b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.111+0000 7ff2cbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2c415ccc0 0x7ff2c4148920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.111+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c40c6890 con 0x7ff2c415ccc0 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.111+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c40c6520 con 0x7ff2c406a5a0 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.111+0000 7ff2cbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c40b3870 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 0x7ff2c40b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 0x7ff2c40b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52108/0 (socket says 172.21.15.84:52108) 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 -- 172.21.15.84:0/463498164 learned_addr learned my addr 172.21.15.84:0/463498164 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c9d9b700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2c406a5a0 0x7ff2c406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2ca59c700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2c415ccc0 0x7ff2c4148920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 -- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2c406a5a0 msgr2=0x7ff2c406d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2c406a5a0 0x7ff2c406d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 -- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2c415ccc0 msgr2=0x7ff2c4148920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2c415ccc0 0x7ff2c4148920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.112+0000 7ff2c959a700 1 -- 172.21.15.84:0/463498164 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2bc009b30 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb879ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcb800d3710 0x7fcb800cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb879ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb800cfa90 0x7fcb800d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb879ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb800cd910 con 0x7fcb800b83c0 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb879ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800cff10 con 0x7fcb800d3710 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb879ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb800d0a80 con 0x7fcb800cfa90 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb85f9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb800cfa90 0x7fcb800d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb85f9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb800cfa90 0x7fcb800d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58676/0 (socket says 172.21.15.84:58676) 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.113+0000 7fcb85f9c700 1 -- 172.21.15.84:0/2309684001 learned_addr learned my addr 172.21.15.84:0/2309684001 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff2c959a700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 0x7ff2c40b3020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c510 tx=0x7ff2b400c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7f442cdff700 1 Processor -- start 2023-07-06T21:39:34.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688004820 con 0x7ff6900d3710 2023-07-06T21:39:34.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688003e30 con 0x7ff6900d3710 2023-07-06T21:39:34.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68801e650 con 0x7ff6900d3710 2023-07-06T21:39:34.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7fb86e19b700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb84c06ad30 0x7fb84c06d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7fcb85f9c700 1 -- 172.21.15.84:0/2309684001 >> v2:172.21.15.84:3300/0 conn(0x7fcb800d3710 msgr2=0x7fcb800cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7fcb85f9c700 1 --2- 172.21.15.84:0/2309684001 >> v2:172.21.15.84:3300/0 conn(0x7fcb800d3710 0x7fcb800cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7fcb85f9c700 1 -- 172.21.15.84:0/2309684001 >> v1:172.21.15.84:6789/0 conn(0x7fcb800b83c0 legacy=0x7fcb800cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:34.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7fcb85f9c700 1 -- 172.21.15.84:0/2309684001 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800b8790 con 0x7fcb800cfa90 2023-07-06T21:39:34.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40052d0 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b4005430 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40038c0 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c40b3b50 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7f442cdff700 1 -- start start 2023-07-06T21:39:34.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.115+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c40b3560 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.115+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f54f4054490 con 0x7f55040add90 2023-07-06T21:39:34.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69015edd0 con 0x7ff6900d3710 2023-07-06T21:39:34.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.114+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69015f390 con 0x7ff6900d3710 2023-07-06T21:39:34.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.116+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700052f0 con 0x7ff6900d3710 2023-07-06T21:39:34.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.117+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac0052f0 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.117+0000 7fcb85f9c700 1 --2- 172.21.15.84:0/2309684001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800cfa90 0x7fcb800d0f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb800b1b00 tx=0x7fcb7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=101473879d080dd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.117+0000 7fb86e19b700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb84c06ad30 0x7fb84c06d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb858001590 tx=0x7fb858007a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.116+0000 7f442cdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f442815ccc0 0x7f4428069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.116+0000 7f442cdff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4428159a60 con 0x7f44280cbcd0 2023-07-06T21:39:34.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.116+0000 7f442cdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442806a810 con 0x7f442806a3d0 2023-07-06T21:39:34.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.116+0000 7f442cdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44280ae180 con 0x7f442815ccc0 2023-07-06T21:39:34.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.117+0000 7f4425d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f442815ccc0 0x7f4428069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.117+0000 7f4425d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f442815ccc0 0x7f4428069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50932/0 (socket says 172.21.15.84:50932) 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.117+0000 7f4425d9b700 1 -- 172.21.15.84:0/2643380004 learned_addr learned my addr 172.21.15.84:0/2643380004 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.118+0000 7f4425d9b700 1 -- 172.21.15.84:0/2643380004 >> v1:172.21.15.107:6789/0 conn(0x7f442806a3d0 legacy=0x7f4428158a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.118+0000 7f4425d9b700 1 -- 172.21.15.84:0/2643380004 >> v1:172.21.15.84:6789/0 conn(0x7f44280cbcd0 legacy=0x7f44280ada70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.118+0000 7f4425d9b700 1 -- 172.21.15.84:0/2643380004 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f442815c1e0 con 0x7f442815ccc0 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.118+0000 7f4425d9b700 1 --2- 172.21.15.84:0/2643380004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f4428069e90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4420013210 tx=0x7f4420013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6fcdc5829ff9a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.119+0000 7f442559a700 1 -- 172.21.15.84:0/2643380004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420016070 con 0x7f442815ccc0 2023-07-06T21:39:34.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.119+0000 7f442559a700 1 -- 172.21.15.84:0/2643380004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420013de0 con 0x7f442815ccc0 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.119+0000 7f442559a700 1 -- 172.21.15.84:0/2643380004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420004250 con 0x7f442815ccc0 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7f442cdff700 1 -- 172.21.15.84:0/2643380004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 msgr2=0x7f4428069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7f442cdff700 1 --2- 172.21.15.84:0/2643380004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f4428069e90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4420013210 tx=0x7f4420013520 comp rx=0 tx=0).stop 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7f442cdff700 1 -- 172.21.15.84:0/2643380004 shutdown_connections 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7f442cdff700 1 --2- 172.21.15.84:0/2643380004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f4428069e90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7f442cdff700 1 -- 172.21.15.84:0/2643380004 >> 172.21.15.84:0/2643380004 conn(0x7f44280cb1c0 msgr2=0x7f44280cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7f442cdff700 1 -- 172.21.15.84:0/2643380004 shutdown_connections 2023-07-06T21:39:34.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.118+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff68801e7b0 con 0x7ff6900d3710 2023-07-06T21:39:34.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7ff68d7fa700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff67c06adb0 0x7ff67c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.120+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff68808c230 con 0x7ff6900d3710 2023-07-06T21:39:34.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.121+0000 7ff694f9b700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff67c06adb0 0x7ff67c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.122+0000 7ff694f9b700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff67c06adb0 0x7ff67c06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff684000f60 tx=0x7ff684007a70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.123+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff688057350 con 0x7ff6900d3710 2023-07-06T21:39:34.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.121+0000 7f442cdff700 1 -- 172.21.15.84:0/2643380004 wait complete. 2023-07-06T21:39:34.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.121+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff2b4003a20 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.122+0000 7ff2ba7fc700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff2a806adb0 0x7ff2a806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.123+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff2b4088be0 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.123+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff2b4053d00 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.124+0000 7f442cdff700 1 Processor -- start 2023-07-06T21:39:34.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.125+0000 7f442cdff700 1 -- start start 2023-07-06T21:39:34.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.127+0000 7fcb77fff700 1 -- 172.21.15.84:0/2309684001 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7800e070 con 0x7fcb800cfa90 2023-07-06T21:39:34.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.127+0000 7fcb77fff700 1 -- 172.21.15.84:0/2309684001 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78003a20 con 0x7fcb800cfa90 2023-07-06T21:39:34.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.127+0000 7fcb77fff700 1 -- 172.21.15.84:0/2309684001 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78003dd0 con 0x7fcb800cfa90 2023-07-06T21:39:34.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.126+0000 7f442cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f442806a3d0 0x7f44281ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.126+0000 7f442cdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44280cbcd0 0x7f44281f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.126+0000 7f442cdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f44281f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.126+0000 7f442cdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44281ffd90 con 0x7f442815ccc0 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.126+0000 7f442cdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428200070 con 0x7f44280cbcd0 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.126+0000 7f442cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4428200350 con 0x7f442806a3d0 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.127+0000 7f4426d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f44281f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.127+0000 7f4426d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f44281f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50946/0 (socket says 172.21.15.84:50946) 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.127+0000 7f4426d9d700 1 -- 172.21.15.84:0/2828431320 learned_addr learned my addr 172.21.15.84:0/2828431320 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4425d9b700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44280cbcd0 0x7f44281f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f442659c700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f442806a3d0 0x7f44281ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4426d9d700 1 -- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f442806a3d0 msgr2=0x7f44281ff2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4426d9d700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f442806a3d0 0x7f44281ff2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4426d9d700 1 -- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44280cbcd0 msgr2=0x7f44281f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4426d9d700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44280cbcd0 0x7f44281f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4426d9d700 1 -- 172.21.15.84:0/2828431320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420011df0 con 0x7f442815ccc0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4425d9b700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44280cbcd0 0x7f44281f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7f4426d9d700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f44281f44e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f441800b780 tx=0x7f441800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.129+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800f840 con 0x7f442815ccc0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.129+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441800f9a0 con 0x7f442815ccc0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.129+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441800fd50 con 0x7f442815ccc0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.129+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4428200630 con 0x7f442815ccc0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.129+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4428157440 con 0x7f442815ccc0 2023-07-06T21:39:34.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.133+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f44180203f0 con 0x7f442815ccc0 2023-07-06T21:39:34.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.135+0000 7ff2c9d9b700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff2a806adb0 0x7ff2a806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.135+0000 7f4416ffd700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f440806adb0 0x7f440806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.135+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4418089840 con 0x7f442815ccc0 2023-07-06T21:39:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.135+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4428200b50 con 0x7f442815ccc0 2023-07-06T21:39:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.136+0000 7ff2c9d9b700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff2a806adb0 0x7ff2a806d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0007fd0 tx=0x7ff2c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7fcb879ff700 1 -- 172.21.15.84:0/2309684001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800cfa90 msgr2=0x7fcb800d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.128+0000 7fcb879ff700 1 --2- 172.21.15.84:0/2309684001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800cfa90 0x7fcb800d0f10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb800b1b00 tx=0x7fcb7800b330 comp rx=0 tx=0).stop 2023-07-06T21:39:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7fcb879ff700 1 -- 172.21.15.84:0/2309684001 shutdown_connections 2023-07-06T21:39:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7fcb879ff700 1 --2- 172.21.15.84:0/2309684001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800cfa90 0x7fcb800d0f10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7fcb879ff700 1 --2- 172.21.15.84:0/2309684001 >> v2:172.21.15.84:3300/0 conn(0x7fcb800d3710 0x7fcb800cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7fcb879ff700 1 -- 172.21.15.84:0/2309684001 >> 172.21.15.84:0/2309684001 conn(0x7fcb8006b350 msgr2=0x7fcb8006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7f442659c700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f440806adb0 0x7f440806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7fcb879ff700 1 -- 172.21.15.84:0/2309684001 shutdown_connections 2023-07-06T21:39:34.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.139+0000 7f442659c700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f440806adb0 0x7f440806d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f441c009910 tx=0x7f441c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.137+0000 7fcb879ff700 1 -- 172.21.15.84:0/2309684001 wait complete. 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.141+0000 7fcb879ff700 1 Processor -- start 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.141+0000 7fcb879ff700 1 -- start start 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.142+0000 7fcb879ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb800b83c0 0x7fcb80202770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.142+0000 7fcb879ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb800cfa90 0x7fcb801fe650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.142+0000 7fcb879ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800d3710 0x7fcb800c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.143+0000 7fcb879ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8015c930 con 0x7fcb800b83c0 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.143+0000 7fcb879ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb801feb90 con 0x7fcb800d3710 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.143+0000 7fcb879ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb801fee60 con 0x7fcb800cfa90 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.144+0000 7fcb84f9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb800cfa90 0x7fcb801fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.144+0000 7fcb8579b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb800b83c0 0x7fcb80202770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.144+0000 7fcb8579b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb800b83c0 0x7fcb80202770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50948/0 (socket says 172.21.15.84:50948) 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.144+0000 7fcb8579b700 1 -- 172.21.15.84:0/949878023 learned_addr learned my addr 172.21.15.84:0/949878023 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:34.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.144+0000 7fcb85f9c700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800d3710 0x7fcb800c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.145+0000 7fcb85f9c700 1 -- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb800cfa90 msgr2=0x7fcb801fe650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.145+0000 7fcb85f9c700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb800cfa90 0x7fcb801fe650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.145+0000 7fcb85f9c700 1 -- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb800b83c0 msgr2=0x7fcb80202770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.145+0000 7fcb85f9c700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb800b83c0 0x7fcb80202770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.145+0000 7fcb85f9c700 1 -- 172.21.15.84:0/949878023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78009b30 con 0x7fcb800d3710 2023-07-06T21:39:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.147+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f44180583f0 con 0x7f442815ccc0 2023-07-06T21:39:34.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.147+0000 7fcb85f9c700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800d3710 0x7fcb800c7420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb78014000 tx=0x7fcb78004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.148+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78004500 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.148+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb78004680 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.148+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78012430 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.148+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb801ff140 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.148+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8015cb80 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.154+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcb78012590 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.155+0000 7fcb75ffb700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb6406adb0 0x7fcb6406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.155+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb7808aa50 con 0x7fcb800d3710 2023-07-06T21:39:34.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.157+0000 7fcb8579b700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb6406adb0 0x7fcb6406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.158+0000 7fcb8579b700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb6406adb0 0x7fcb6406d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb800c6f40 tx=0x7fcb7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.162+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb600052f0 con 0x7fcb800d3710 2023-07-06T21:39:34.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.168+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcb78055b70 con 0x7fcb800d3710 2023-07-06T21:39:34.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.531+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efec01ff7f0 con 0x7efec00c52f0 2023-07-06T21:39:34.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.532+0000 7efea77fe700 1 -- 172.21.15.84:0/3597798026 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efec01ff7f0 con 0x7efec00c52f0 2023-07-06T21:39:34.538 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-07-06T21:39:34.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.536+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efeac06ad30 msgr2=0x7efeac06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.536+0000 7efec57ff700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efeac06ad30 0x7efeac06d1e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb001c040 comp rx=0 tx=0).stop 2023-07-06T21:39:34.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.536+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 msgr2=0x7efec006dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.536+0000 7efec57ff700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 0x7efec006dc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efeb400c4f0 tx=0x7efeb400c800 comp rx=0 tx=0).stop 2023-07-06T21:39:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.537+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 shutdown_connections 2023-07-06T21:39:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.537+0000 7efec57ff700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efeac06ad30 0x7efeac06d1e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.537+0000 7efec57ff700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efec01608e0 0x7efec006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.537+0000 7efec57ff700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efec00c4dd0 0x7efec0201e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.537+0000 7efec57ff700 1 --2- 172.21.15.84:0/3597798026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efec00c52f0 0x7efec006dc10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.537+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 >> 172.21.15.84:0/3597798026 conn(0x7efec00cb1c0 msgr2=0x7efec00cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.538+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 shutdown_connections 2023-07-06T21:39:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.538+0000 7efec57ff700 1 -- 172.21.15.84:0/3597798026 wait complete. 2023-07-06T21:39:34.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.554+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f920015f160 con 0x7f92001f3b10 2023-07-06T21:39:34.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.555+0000 7f91f7fff700 1 -- 172.21.15.84:0/2647628027 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f920015f160 con 0x7f92001f3b10 2023-07-06T21:39:34.561 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-07-06T21:39:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.559+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91e806ade0 msgr2=0x7f91e806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.559+0000 7f9205dff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91e806ade0 0x7f91e806d290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f920006a950 tx=0x7f91f8009030 comp rx=0 tx=0).stop 2023-07-06T21:39:34.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.559+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 msgr2=0x7f92001f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.559+0000 7f9205dff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 0x7f92001f3f80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f920006b760 tx=0x7f91ec0117e0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.560+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 shutdown_connections 2023-07-06T21:39:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.560+0000 7f9205dff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91e806ade0 0x7f91e806d290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.560+0000 7f9205dff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92001f3b10 0x7f92001f3f80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.560+0000 7f9205dff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92001fa140 0x7f920015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.560+0000 7f9205dff700 1 --2- 172.21.15.84:0/2647628027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92001fa510 0x7f92001f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.560+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 >> 172.21.15.84:0/2647628027 conn(0x7f92000cb1c0 msgr2=0x7f920015aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.563+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 shutdown_connections 2023-07-06T21:39:34.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.563+0000 7f9205dff700 1 -- 172.21.15.84:0/2647628027 wait complete. 2023-07-06T21:39:34.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.563+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f54ec005870 con 0x7f55040add90 2023-07-06T21:39:34.571 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-07-06T21:39:34.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.565+0000 7f54fa7fc700 1 -- 172.21.15.84:0/3019291093 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f54f4057ab0 con 0x7f55040add90 2023-07-06T21:39:34.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.3 2023-07-06T21:39:34.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54e806ad30 msgr2=0x7f54e806d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54e806ad30 0x7f54e806d1e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f55040c9f90 tx=0x7f5500008040 comp rx=0 tx=0).stop 2023-07-06T21:39:34.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 msgr2=0x7f55040c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 0x7f55040c7bc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f54f400b4a0 tx=0x7f54f400b860 comp rx=0 tx=0).stop 2023-07-06T21:39:34.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb86806dab0 con 0x7fb86815ccc0 2023-07-06T21:39:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 shutdown_connections 2023-07-06T21:39:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54e806ad30 0x7f54e806d1e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55040c91c0 0x7f55040af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5504069e90 0x7f55040c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 --2- 172.21.15.84:0/3019291093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f55040add90 0x7f55040c7bc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.570+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 >> 172.21.15.84:0/3019291093 conn(0x7f55040cb1c0 msgr2=0x7f55040ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.571+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 shutdown_connections 2023-07-06T21:39:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.571+0000 7f550bdff700 1 -- 172.21.15.84:0/3019291093 wait complete. 2023-07-06T21:39:34.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.571+0000 7fb85effd700 1 -- 172.21.15.84:0/2913265708 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb86806dab0 con 0x7fb86815ccc0 2023-07-06T21:39:34.577 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-07-06T21:39:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb84c06ad30 msgr2=0x7fb84c06d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb84c06ad30 0x7fb84c06d1e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb858001590 tx=0x7fb858007a20 comp rx=0 tx=0).stop 2023-07-06T21:39:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 msgr2=0x7fb8681514f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 0x7fb8681514f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb860004000 tx=0x7fb860004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff670005870 con 0x7ff6900d3710 2023-07-06T21:39:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 shutdown_connections 2023-07-06T21:39:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb84c06ad30 0x7fb84c06d1e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb86815ccc0 0x7fb8681514f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb86806a5a0 0x7fb8680b2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 --2- 172.21.15.84:0/2913265708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8680cbcd0 0x7fb8680b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.575+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 >> 172.21.15.84:0/2913265708 conn(0x7fb8680cb1c0 msgr2=0x7fb8680ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.580 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-07-06T21:39:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.576+0000 7ff68d7fa700 1 -- 172.21.15.84:0/323866487 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff68805a970 con 0x7ff6900d3710 2023-07-06T21:39:34.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.576+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 shutdown_connections 2023-07-06T21:39:34.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.576+0000 7fb8703ff700 1 -- 172.21.15.84:0/2913265708 wait complete. 2023-07-06T21:39:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff67c06adb0 msgr2=0x7ff67c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff67c06adb0 0x7ff67c06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff684000f60 tx=0x7ff684007a70 comp rx=0 tx=0).stop 2023-07-06T21:39:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900d3710 msgr2=0x7ff6900ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900d3710 0x7ff6900ca6c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff6900cf520 tx=0x7ff688004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 shutdown_connections 2023-07-06T21:39:34.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff67c06adb0 0x7ff67c06d260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6900d3710 0x7ff6900ca6c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6900b83c0 0x7ff6900b7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 --2- 172.21.15.84:0/323866487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6900cfa90 0x7ff690071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 >> 172.21.15.84:0/323866487 conn(0x7ff69006b350 msgr2=0x7ff6900d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.580+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 shutdown_connections 2023-07-06T21:39:34.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.581+0000 7ff6971ff700 1 -- 172.21.15.84:0/323866487 wait complete. 2023-07-06T21:39:34.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.590+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ff2ac005870 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.591+0000 7ff2ba7fc700 1 -- 172.21.15.84:0/463498164 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff2b4057320 con 0x7ff2c40cbcd0 2023-07-06T21:39:34.594 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-07-06T21:39:34.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.594+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f44280cd690 con 0x7f442815ccc0 2023-07-06T21:39:34.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.595+0000 7f4416ffd700 1 -- 172.21.15.84:0/2828431320 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4418057f80 con 0x7f442815ccc0 2023-07-06T21:39:34.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.4 2023-07-06T21:39:34.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.595+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff2a806adb0 msgr2=0x7ff2a806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.595+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff2a806adb0 0x7ff2a806d260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0007fd0 tx=0x7ff2c0009040 comp rx=0 tx=0).stop 2023-07-06T21:39:34.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.595+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 msgr2=0x7ff2c40b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.595+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 0x7ff2c40b3020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400c510 tx=0x7ff2b400c820 comp rx=0 tx=0).stop 2023-07-06T21:39:34.601 INFO:teuthology.orchestra.run.smithi084.stdout:55834574853 2023-07-06T21:39:34.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 shutdown_connections 2023-07-06T21:39:34.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff2a806adb0 0x7ff2a806d260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2c415ccc0 0x7ff2c4148920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2c406a5a0 0x7ff2c406d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 --2- 172.21.15.84:0/463498164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2c40cbcd0 0x7ff2c40b3020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 >> 172.21.15.84:0/463498164 conn(0x7ff2c40cb1c0 msgr2=0x7ff2c41492a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fcb600055a0 con 0x7fcb800d3710 2023-07-06T21:39:34.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 shutdown_connections 2023-07-06T21:39:34.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.597+0000 7ff2cbfff700 1 -- 172.21.15.84:0/463498164 wait complete. 2023-07-06T21:39:34.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.598+0000 7fcb75ffb700 1 -- 172.21.15.84:0/949878023 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcb7802b090 con 0x7fcb800d3710 2023-07-06T21:39:34.604 INFO:teuthology.orchestra.run.smithi084.stdout:55834574852 2023-07-06T21:39:34.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.598+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f440806adb0 msgr2=0x7f440806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f440806adb0 0x7f440806d260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f441c009910 tx=0x7f441c008040 comp rx=0 tx=0).stop 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 msgr2=0x7f44281f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f44281f44e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f441800b780 tx=0x7f441800bb40 comp rx=0 tx=0).stop 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 shutdown_connections 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f440806adb0 0x7f440806d260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f442815ccc0 0x7f44281f44e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f442806a3d0 0x7f44281ff2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 --2- 172.21.15.84:0/2828431320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44280cbcd0 0x7f44281f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.599+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 >> 172.21.15.84:0/2828431320 conn(0x7f44280cb1c0 msgr2=0x7f44280af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.600+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 shutdown_connections 2023-07-06T21:39:34.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.600+0000 7f442cdff700 1 -- 172.21.15.84:0/2828431320 wait complete. 2023-07-06T21:39:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb6406adb0 msgr2=0x7fcb6406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb6406adb0 0x7fcb6406d260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcb800c6f40 tx=0x7fcb7c008040 comp rx=0 tx=0).stop 2023-07-06T21:39:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800d3710 msgr2=0x7fcb800c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800d3710 0x7fcb800c7420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb78014000 tx=0x7fcb78004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 shutdown_connections 2023-07-06T21:39:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcb6406adb0 0x7fcb6406d260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb800d3710 0x7fcb800c7420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb800b83c0 0x7fcb80202770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 --2- 172.21.15.84:0/949878023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb800cfa90 0x7fcb801fe650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:34.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.601+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 >> 172.21.15.84:0/949878023 conn(0x7fcb8006b350 msgr2=0x7fcb800d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:34.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.603+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 shutdown_connections 2023-07-06T21:39:34.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:34.603+0000 7fcb879ff700 1 -- 172.21.15.84:0/949878023 wait complete. 2023-07-06T21:39:34.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.0 2023-07-06T21:39:34.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.1 2023-07-06T21:39:34.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.6 2023-07-06T21:39:34.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.5 2023-07-06T21:39:34.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.2 2023-07-06T21:39:34.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574852 for osd.7 2023-07-06T21:39:35.574 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-07-06T21:39:35.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-07-06T21:39:35.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-07-06T21:39:35.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-07-06T21:39:35.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-07-06T21:39:35.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-07-06T21:39:35.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-07-06T21:39:35.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-07-06T21:39:35.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.888+0000 7f7fbbfff700 1 Processor -- start 2023-07-06T21:39:35.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.888+0000 7f7fbbfff700 1 -- start start 2023-07-06T21:39:35.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.890+0000 7f7fbbfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7fbc158350 0x7f7fbc0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.890+0000 7f7fbbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fbc0ae770 0x7f7fbc0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.890+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc0aebc0 0x7f7fbc0cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.890+0000 7f7fbbfff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc0ada60 con 0x7f7fbc0ae770 2023-07-06T21:39:35.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.890+0000 7f7fbbfff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc0af6e0 con 0x7f7fbc158350 2023-07-06T21:39:35.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.890+0000 7f7fbbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0ae2e0 con 0x7f7fbc0aebc0 2023-07-06T21:39:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.891+0000 7f7fba7fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fbc0ae770 0x7f7fbc0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.891+0000 7f7fba7fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7fbc0ae770 0x7f7fbc0caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50958/0 (socket says 172.21.15.84:50958) 2023-07-06T21:39:35.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.891+0000 7f7fba7fc700 1 -- 172.21.15.84:0/2112707836 learned_addr learned my addr 172.21.15.84:0/2112707836 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.892+0000 7f7fba7fc700 1 -- 172.21.15.84:0/2112707836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc0aebc0 msgr2=0x7f7fbc0cb470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.892+0000 7f7fba7fc700 1 --2- 172.21.15.84:0/2112707836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc0aebc0 0x7f7fbc0cb470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.892+0000 7f7fba7fc700 1 -- 172.21.15.84:0/2112707836 >> v2:172.21.15.107:3300/0 conn(0x7f7fbc158350 msgr2=0x7f7fbc0af1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.892+0000 7f7fba7fc700 1 --2- 172.21.15.84:0/2112707836 >> v2:172.21.15.107:3300/0 conn(0x7f7fbc158350 0x7f7fbc0af1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.892+0000 7f7fba7fc700 1 -- 172.21.15.84:0/2112707836 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc0afc80 con 0x7f7fbc0ae770 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.893+0000 7f7fba7fc700 1 --2- 172.21.15.84:0/2112707836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0caf30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc06ff60 tx=0x7f7fb400d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7896a51572077ca7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:35.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.894+0000 7f7fb9ffb700 1 -- 172.21.15.84:0/2112707836 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4007d30 con 0x7f7fbc0ae770 2023-07-06T21:39:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.894+0000 7f7fb9ffb700 1 -- 172.21.15.84:0/2112707836 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb4007e90 con 0x7f7fbc0ae770 2023-07-06T21:39:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.894+0000 7f7fb9ffb700 1 -- 172.21.15.84:0/2112707836 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4012710 con 0x7f7fbc0ae770 2023-07-06T21:39:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.895+0000 7f7fbbfff700 1 -- 172.21.15.84:0/2112707836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 msgr2=0x7f7fbc0caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.895+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/2112707836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0caf30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc06ff60 tx=0x7f7fb400d650 comp rx=0 tx=0).stop 2023-07-06T21:39:35.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 -- 172.21.15.84:0/2112707836 shutdown_connections 2023-07-06T21:39:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/2112707836 >> v2:172.21.15.107:3300/0 conn(0x7f7fbc158350 0x7f7fbc0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/2112707836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc0aebc0 0x7f7fbc0cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/2112707836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0caf30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 -- 172.21.15.84:0/2112707836 >> 172.21.15.84:0/2112707836 conn(0x7f7fbc0ce520 msgr2=0x7f7fbc0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:35.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 -- 172.21.15.84:0/2112707836 shutdown_connections 2023-07-06T21:39:35.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.896+0000 7f7fbbfff700 1 -- 172.21.15.84:0/2112707836 wait complete. 2023-07-06T21:39:35.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.900+0000 7f7fbbfff700 1 Processor -- start 2023-07-06T21:39:35.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.900+0000 7f7fbbfff700 1 -- start start 2023-07-06T21:39:35.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.902+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.902+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fbc0aebc0 0x7f7fbc1f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.902+0000 7f7fbbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc158350 0x7f7fbc156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.902+0000 7f7fbbfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0b1890 con 0x7f7fbc0ae770 2023-07-06T21:39:35.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.902+0000 7f7fbbfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc1f1f30 con 0x7f7fbc0aebc0 2023-07-06T21:39:35.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.902+0000 7f7fbbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc0b2360 con 0x7f7fbc158350 2023-07-06T21:39:35.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.903+0000 7f7fbaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.903+0000 7f7fbaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50972/0 (socket says 172.21.15.84:50972) 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.903+0000 7f7fbaffd700 1 -- 172.21.15.84:0/1173120734 learned_addr learned my addr 172.21.15.84:0/1173120734 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.904+0000 7f7fbaffd700 1 -- 172.21.15.84:0/1173120734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc158350 msgr2=0x7f7fbc156b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.904+0000 7f7fbaffd700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc158350 0x7f7fbc156b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.904+0000 7f7fbaffd700 1 -- 172.21.15.84:0/1173120734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fbc0aebc0 msgr2=0x7f7fbc1f19f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.904+0000 7f7fbaffd700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fbc0aebc0 0x7f7fbc1f19f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.904+0000 7f7fbaffd700 1 -- 172.21.15.84:0/1173120734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb400d050 con 0x7f7fbc0ae770 2023-07-06T21:39:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.905+0000 7f7fbaffd700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0b1350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00b450 tx=0x7f7fac00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.906+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac011830 con 0x7f7fbc0ae770 2023-07-06T21:39:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.906+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac0119b0 con 0x7f7fbc0ae770 2023-07-06T21:39:35.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.906+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac011d60 con 0x7f7fbc0ae770 2023-07-06T21:39:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.906+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc0b1ac0 con 0x7f7fbc0ae770 2023-07-06T21:39:35.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.907+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc0b2080 con 0x7f7fbc0ae770 2023-07-06T21:39:35.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.914+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7fac01a400 con 0x7f7fbc0ae770 2023-07-06T21:39:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.916+0000 7f7fab7fe700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7fa406af10 0x7f7fa406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.916+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7fac0894e0 con 0x7f7fbc0ae770 2023-07-06T21:39:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.917+0000 7f7fba7fc700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7fa406af10 0x7f7fa406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.917+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f980052f0 con 0x7f7fbc0ae770 2023-07-06T21:39:35.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.918+0000 7f7fba7fc700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7fa406af10 0x7f7fa406d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400d690 tx=0x7f7fb4015000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:35.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.928+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7fac054570 con 0x7f7fbc0ae770 2023-07-06T21:39:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.952+0000 7f2fc75ff700 1 Processor -- start 2023-07-06T21:39:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.952+0000 7f2fc75ff700 1 -- start start 2023-07-06T21:39:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.954+0000 7f2fc75ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fc00c91c0 0x7f2fc0070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.954+0000 7f2fc75ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2fc006a5a0 0x7f2fc00c4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.954+0000 7f2fc75ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc0071180 con 0x7f2fc00ccc90 2023-07-06T21:39:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.954+0000 7f2fc75ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc016cae0 con 0x7f2fc00c91c0 2023-07-06T21:39:35.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.954+0000 7f2fc75ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00c3a40 con 0x7f2fc006a5a0 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.955+0000 7f2fc539b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fc00c91c0 0x7f2fc0070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.955+0000 7f2fc539b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2fc00c91c0 0x7f2fc0070c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50988/0 (socket says 172.21.15.84:50988) 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.955+0000 7f2fc539b700 1 -- 172.21.15.84:0/577434735 learned_addr learned my addr 172.21.15.84:0/577434735 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.956+0000 7f2fc539b700 1 -- 172.21.15.84:0/577434735 >> v2:172.21.15.107:3300/0 conn(0x7f2fc006a5a0 msgr2=0x7f2fc00c4770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.956+0000 7f2fc539b700 1 --2- 172.21.15.84:0/577434735 >> v2:172.21.15.107:3300/0 conn(0x7f2fc006a5a0 0x7f2fc00c4770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.956+0000 7f2fc539b700 1 -- 172.21.15.84:0/577434735 >> v1:172.21.15.107:6789/0 conn(0x7f2fc00ccc90 legacy=0x7f2fc0069e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.956+0000 7f2fc539b700 1 -- 172.21.15.84:0/577434735 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc00c9590 con 0x7f2fc00c91c0 2023-07-06T21:39:35.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.956+0000 7f2fc539b700 1 --2- 172.21.15.84:0/577434735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc00c91c0 0x7f2fc0070c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00ce360 tx=0x7f2fb00134c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdb5b61952d09aa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.957+0000 7f2fb7fff700 1 -- 172.21.15.84:0/577434735 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0013db0 con 0x7f2fc00c91c0 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.957+0000 7f2fb7fff700 1 -- 172.21.15.84:0/577434735 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb0004030 con 0x7f2fc00c91c0 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.957+0000 7f2fb7fff700 1 -- 172.21.15.84:0/577434735 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00043c0 con 0x7f2fc00c91c0 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.958+0000 7f2fc75ff700 1 -- 172.21.15.84:0/577434735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc00c91c0 msgr2=0x7f2fc0070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.958+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/577434735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc00c91c0 0x7f2fc0070c40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00ce360 tx=0x7f2fb00134c0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.958+0000 7f2fc75ff700 1 -- 172.21.15.84:0/577434735 shutdown_connections 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.958+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/577434735 >> v2:172.21.15.107:3300/0 conn(0x7f2fc006a5a0 0x7f2fc00c4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.986 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.958+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/577434735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc00c91c0 0x7f2fc0070c40 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2fc00ce360 tx=0x7f2fb00134c0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.958+0000 7f2fc75ff700 1 -- 172.21.15.84:0/577434735 >> 172.21.15.84:0/577434735 conn(0x7f2fc00cb1c0 msgr2=0x7f2fc00cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.960+0000 7f2fc75ff700 1 -- 172.21.15.84:0/577434735 shutdown_connections 2023-07-06T21:39:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.960+0000 7f26499ff700 1 Processor -- start 2023-07-06T21:39:35.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.960+0000 7f2fc75ff700 1 -- 172.21.15.84:0/577434735 wait complete. 2023-07-06T21:39:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.963+0000 7f2fc75ff700 1 Processor -- start 2023-07-06T21:39:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.961+0000 7f26499ff700 1 -- start start 2023-07-06T21:39:35.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.963+0000 7f26499ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2644160830 0x7f26440adae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.963+0000 7f26499ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f264406a5a0 0x7f2644160c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.963+0000 7f26499ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26440ae020 con 0x7f26440c3600 2023-07-06T21:39:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.963+0000 7f26499ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26440cd030 con 0x7f264406a5a0 2023-07-06T21:39:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.963+0000 7f26499ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2644164370 con 0x7f2644160830 2023-07-06T21:39:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.964+0000 7f2642ffd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2644160830 0x7f26440adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.964+0000 7f3de31ff700 1 Processor -- start 2023-07-06T21:39:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2642ffd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2644160830 0x7f26440adae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58742/0 (socket says 172.21.15.84:58742) 2023-07-06T21:39:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2642ffd700 1 -- 172.21.15.84:0/464984232 learned_addr learned my addr 172.21.15.84:0/464984232 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:35.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.964+0000 7f2fc75ff700 1 -- start start 2023-07-06T21:39:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2fc75ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc006a5a0 0x7f2fc00b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2fc75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc00ccc90 0x7f2fc00c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2fc75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 0x7f2fc0161180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2fc75ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01f33d0 con 0x7f2fc006a5a0 2023-07-06T21:39:35.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2fc75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01f36b0 con 0x7f2fc00ccc90 2023-07-06T21:39:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f2fc75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c7280 con 0x7f2fc00b4370 2023-07-06T21:39:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f2fc5b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 0x7f2fc0161180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f2fc5b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 0x7f2fc0161180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52156/0 (socket says 172.21.15.84:52156) 2023-07-06T21:39:35.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f2fc5b9c700 1 -- 172.21.15.84:0/3071385681 learned_addr learned my addr 172.21.15.84:0/3071385681 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f2642ffd700 1 -- 172.21.15.84:0/464984232 >> v2:172.21.15.84:3300/0 conn(0x7f264406a5a0 msgr2=0x7f2644160c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f2642ffd700 1 --2- 172.21.15.84:0/464984232 >> v2:172.21.15.84:3300/0 conn(0x7f264406a5a0 0x7f2644160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f2642ffd700 1 -- 172.21.15.84:0/464984232 >> v1:172.21.15.84:6789/0 conn(0x7f26440c3600 legacy=0x7f2644069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f2642ffd700 1 -- 172.21.15.84:0/464984232 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644164650 con 0x7f2644160830 2023-07-06T21:39:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f26427fc700 1 -- 172.21.15.84:0/464984232 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790384715 0 0) 0x7f26440ae020 con 0x7f26440c3600 2023-07-06T21:39:35.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.965+0000 7f3de31ff700 1 -- start start 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f3de31ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ddc15ccc0 0x7f3ddc1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f3de31ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc159a60 con 0x7f3ddc0adce0 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f3de31ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc159780 con 0x7f3ddc06a5a0 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.967+0000 7f3de31ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc15c1e0 con 0x7f3ddc15ccc0 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f3de179c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ddc15ccc0 0x7f3ddc1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f3de179c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ddc15ccc0 0x7f3ddc1591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51004/0 (socket says 172.21.15.84:51004) 2023-07-06T21:39:35.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.968+0000 7f3de179c700 1 -- 172.21.15.84:0/3324793233 learned_addr learned my addr 172.21.15.84:0/3324793233 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f2642ffd700 1 --2- 172.21.15.84:0/464984232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2644160830 0x7f26440adae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26440cb120 tx=0x7f263400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=538c3a976885d297 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:35.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f2fc5b9c700 1 -- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc00ccc90 msgr2=0x7f2fc00c7a80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-06T21:39:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f2fc5b9c700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc00ccc90 0x7f2fc00c7a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f2fc5b9c700 1 -- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc006a5a0 msgr2=0x7f2fc00b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f2fc5b9c700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc006a5a0 0x7f2fc00b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f2fc5b9c700 1 -- 172.21.15.84:0/3071385681 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0011df0 con 0x7f2fc00b4370 2023-07-06T21:39:35.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f26427fc700 1 -- 172.21.15.84:0/464984232 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400e070 con 0x7f2644160830 2023-07-06T21:39:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f26427fc700 1 -- 172.21.15.84:0/464984232 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263400b9c0 con 0x7f2644160830 2023-07-06T21:39:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f26427fc700 1 -- 172.21.15.84:0/464984232 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400bd70 con 0x7f2644160830 2023-07-06T21:39:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f3ddb7fe700 1 -- 172.21.15.84:0/3324793233 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562131981 0 0) 0x7f3ddc159780 con 0x7f3ddc06a5a0 2023-07-06T21:39:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f3de179c700 1 -- 172.21.15.84:0/3324793233 >> v1:172.21.15.107:6789/0 conn(0x7f3ddc06a5a0 legacy=0x7f3ddc158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.969+0000 7f3de179c700 1 -- 172.21.15.84:0/3324793233 >> v1:172.21.15.84:6789/0 conn(0x7f3ddc0adce0 legacy=0x7f3ddc069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:35.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f3de179c700 1 -- 172.21.15.84:0/3324793233 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc15c4c0 con 0x7f3ddc15ccc0 2023-07-06T21:39:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f26499ff700 1 -- 172.21.15.84:0/464984232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2644160830 msgr2=0x7f26440adae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f26499ff700 1 --2- 172.21.15.84:0/464984232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2644160830 0x7f26440adae0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f26440cb120 tx=0x7f263400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f26499ff700 1 -- 172.21.15.84:0/464984232 shutdown_connections 2023-07-06T21:39:35.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f26499ff700 1 --2- 172.21.15.84:0/464984232 >> v2:172.21.15.84:3300/0 conn(0x7f264406a5a0 0x7f2644160c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f26499ff700 1 --2- 172.21.15.84:0/464984232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2644160830 0x7f26440adae0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:35.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f26499ff700 1 -- 172.21.15.84:0/464984232 >> 172.21.15.84:0/464984232 conn(0x7f26440cf3d0 msgr2=0x7f26440cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:35.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f3de179c700 1 --2- 172.21.15.84:0/3324793233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 0x7f3ddc1591a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d370 tx=0x7f3dd400d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa18075576ca607 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:35.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.970+0000 7f2fc5b9c700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 0x7f2fc0161180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b780 tx=0x7f2fb800bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f26499ff700 1 -- 172.21.15.84:0/464984232 shutdown_connections 2023-07-06T21:39:36.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f26499ff700 1 -- 172.21.15.84:0/464984232 wait complete. 2023-07-06T21:39:36.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f3ddb7fe700 1 -- 172.21.15.84:0/3324793233 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4010070 con 0x7f3ddc15ccc0 2023-07-06T21:39:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f3ddb7fe700 1 -- 172.21.15.84:0/3324793233 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400dcd0 con 0x7f3ddc15ccc0 2023-07-06T21:39:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f3ddb7fe700 1 -- 172.21.15.84:0/3324793233 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4007d50 con 0x7f3ddc15ccc0 2023-07-06T21:39:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.972+0000 7f3de31ff700 1 -- 172.21.15.84:0/3324793233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 msgr2=0x7f3ddc1591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.972+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3324793233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 0x7f3ddc1591a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400d370 tx=0x7f3dd400d680 comp rx=0 tx=0).stop 2023-07-06T21:39:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800f840 con 0x7f2fc00b4370 2023-07-06T21:39:36.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb800f9c0 con 0x7f2fc00b4370 2023-07-06T21:39:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.971+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb800fd70 con 0x7f2fc00b4370 2023-07-06T21:39:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.973+0000 7f3de31ff700 1 -- 172.21.15.84:0/3324793233 shutdown_connections 2023-07-06T21:39:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.973+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3324793233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 0x7f3ddc1591a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.973+0000 7f3de31ff700 1 -- 172.21.15.84:0/3324793233 >> 172.21.15.84:0/3324793233 conn(0x7f3ddc0cb1c0 msgr2=0x7f3ddc0cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.973+0000 7f3de31ff700 1 -- 172.21.15.84:0/3324793233 shutdown_connections 2023-07-06T21:39:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.972+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc00c7560 con 0x7f2fc00b4370 2023-07-06T21:39:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.972+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc01f3990 con 0x7f2fc00b4370 2023-07-06T21:39:36.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.973+0000 7f26499ff700 1 Processor -- start 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.974+0000 7f26499ff700 1 -- start start 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f26499ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 0x7f2644141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f26499ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26440c3600 0x7f264406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f26499ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2644160830 0x7f264406c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f26499ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2644143030 con 0x7f2644160830 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f26499ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2644142690 con 0x7f264406a5a0 2023-07-06T21:39:36.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f26499ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2644142970 con 0x7f26440c3600 2023-07-06T21:39:36.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.976+0000 7f2642ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 0x7f2644141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.976+0000 7f263a7fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26440c3600 0x7f264406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.973+0000 7f3de31ff700 1 -- 172.21.15.84:0/3324793233 wait complete. 2023-07-06T21:39:36.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.976+0000 7f3de31ff700 1 Processor -- start 2023-07-06T21:39:36.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.976+0000 7f2642ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 0x7f2644141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58752/0 (socket says 172.21.15.84:58752) 2023-07-06T21:39:36.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.976+0000 7f2642ffd700 1 -- 172.21.15.84:0/2414440646 learned_addr learned my addr 172.21.15.84:0/2414440646 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.974+0000 7f0a805ff700 1 Processor -- start 2023-07-06T21:39:36.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.975+0000 7f0a805ff700 1 -- start start 2023-07-06T21:39:36.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f0a805ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0a781608e0 0x7f0a78069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f0a805ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a7806a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f2642ffd700 1 -- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26440c3600 msgr2=0x7f264406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f2642ffd700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26440c3600 0x7f264406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f2642ffd700 1 -- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2644160830 msgr2=0x7f264406c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f2642ffd700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2644160830 0x7f264406c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f2642ffd700 1 -- 172.21.15.84:0/2414440646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2634009b30 con 0x7f264406a5a0 2023-07-06T21:39:36.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f0a805ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a7806e5f0 0x7f0a78070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f0a805ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a780c91c0 con 0x7f0a7806e5f0 2023-07-06T21:39:36.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f0a805ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a78164890 con 0x7f0a781608e0 2023-07-06T21:39:36.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f0a805ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78164b70 con 0x7f0a780c52f0 2023-07-06T21:39:36.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f2642ffd700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 0x7f2644141d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f263400b730 tx=0x7f2634004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.977+0000 7f3de31ff700 1 -- start start 2023-07-06T21:39:36.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f0a7e39b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0a781608e0 0x7f0a78069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f0a7e39b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0a781608e0 0x7f0a78069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58766/0 (socket says 172.21.15.84:58766) 2023-07-06T21:39:36.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f0a7e39b700 1 -- 172.21.15.84:0/3796693171 learned_addr learned my addr 172.21.15.84:0/3796693171 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f0a7eb9c700 1 --2- 172.21.15.84:0/3796693171 >> v2:172.21.15.84:3300/0 conn(0x7f0a7806e5f0 0x7f0a78070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f0a7db9a700 1 --2- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a7806a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26340092e0 con 0x7f264406a5a0 2023-07-06T21:39:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26340099b0 con 0x7f264406a5a0 2023-07-06T21:39:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263401b670 con 0x7f264406a5a0 2023-07-06T21:39:36.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.978+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2644142c50 con 0x7f264406a5a0 2023-07-06T21:39:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.979+0000 7f0a7e39b700 1 -- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 msgr2=0x7f0a7806a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.979+0000 7f0a7e39b700 1 --2- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a7806a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.979+0000 7f0a7e39b700 1 -- 172.21.15.84:0/3796693171 >> v2:172.21.15.84:3300/0 conn(0x7f0a7806e5f0 msgr2=0x7f0a78070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.979+0000 7f0a7e39b700 1 --2- 172.21.15.84:0/3796693171 >> v2:172.21.15.84:3300/0 conn(0x7f0a7806e5f0 0x7f0a78070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.979+0000 7f0a7e39b700 1 -- 172.21.15.84:0/3796693171 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a78164e50 con 0x7f0a781608e0 2023-07-06T21:39:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.979+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2644142290 con 0x7f264406a5a0 2023-07-06T21:39:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.980+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26440cbe20 con 0x7f264406a5a0 2023-07-06T21:39:36.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.980+0000 7f0a7e39b700 1 --2- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a781608e0 0x7f0a78069e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a780b4950 tx=0x7f0a74009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96db6b2be346e501 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.980+0000 7f3de31ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ddc06a5a0 0x7f3ddc0af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.980+0000 7f3de31ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ddc0adce0 0x7f3ddc157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.980+0000 7f3de31ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 0x7f3ddc158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3de31ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0c6470 con 0x7f3ddc15ccc0 2023-07-06T21:39:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3de31ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1e28c0 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3de31ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc1e1f90 con 0x7f3ddc0adce0 2023-07-06T21:39:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ddc0adce0 0x7f3ddc157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3ddbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ddc0adce0 0x7f3ddc157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52190/0 (socket says 172.21.15.84:52190) 2023-07-06T21:39:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3ddbfff700 1 -- 172.21.15.84:0/3310604901 learned_addr learned my addr 172.21.15.84:0/3310604901 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc00c7780 con 0x7f2fc00b4370 2023-07-06T21:39:36.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f0a7d399700 1 -- 172.21.15.84:0/3796693171 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74011830 con 0x7f0a781608e0 2023-07-06T21:39:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f0a7d399700 1 -- 172.21.15.84:0/3796693171 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74011990 con 0x7f0a781608e0 2023-07-06T21:39:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f0a7d399700 1 -- 172.21.15.84:0/3796693171 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74011d20 con 0x7f0a781608e0 2023-07-06T21:39:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.981+0000 7f3de0f9b700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ddc06a5a0 0x7f3ddc0af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f263401b7d0 con 0x7f264406a5a0 2023-07-06T21:39:36.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f3de0f9b700 1 -- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ddc0adce0 msgr2=0x7f3ddc157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f3de0f9b700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ddc0adce0 0x7f3ddc157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f3de0f9b700 1 -- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 msgr2=0x7f3ddc158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f3de0f9b700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 0x7f3ddc158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f3de0f9b700 1 -- 172.21.15.84:0/3310604901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd400d050 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f0a805ff700 1 -- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a781608e0 msgr2=0x7f0a78069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.982+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a781608e0 0x7f0a78069e90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0a780b4950 tx=0x7f0a74009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f3de0f9b700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ddc06a5a0 0x7f3ddc0af4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c400 tx=0x7f3dcc00c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2fb80203f0 con 0x7f2fc00b4370 2023-07-06T21:39:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f0a805ff700 1 -- 172.21.15.84:0/3796693171 shutdown_connections 2023-07-06T21:39:36.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a781608e0 0x7f0a78069e90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3796693171 >> v2:172.21.15.84:3300/0 conn(0x7f0a7806e5f0 0x7f0a78070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3796693171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a7806a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.983+0000 7f0a805ff700 1 -- 172.21.15.84:0/3796693171 >> 172.21.15.84:0/3796693171 conn(0x7f0a780cb1c0 msgr2=0x7f0a780cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.984+0000 7f0a805ff700 1 -- 172.21.15.84:0/3796693171 shutdown_connections 2023-07-06T21:39:36.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.984+0000 7f2640ff9700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f262406adb0 0x7f262406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.984+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f263408abf0 con 0x7f264406a5a0 2023-07-06T21:39:36.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.985+0000 7f263a7fc700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f262406adb0 0x7f262406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.985+0000 7f2fb5ffb700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2fa406adb0 0x7f2fa406d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.985+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2fb8089830 con 0x7f2fc00b4370 2023-07-06T21:39:36.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.984+0000 7f0a805ff700 1 -- 172.21.15.84:0/3796693171 wait complete. 2023-07-06T21:39:36.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.986+0000 7f2fc539b700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2fa406adb0 0x7f2fa406d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.986+0000 7f263a7fc700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f262406adb0 0x7f262406d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2644163220 tx=0x7f262c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.986+0000 7f0a805ff700 1 Processor -- start 2023-07-06T21:39:36.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.987+0000 7f0a805ff700 1 -- start start 2023-07-06T21:39:36.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.987+0000 7f2fc539b700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2fa406adb0 0x7f2fa406d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2fc006b720 tx=0x7f2fb0009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011790 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc0118f0 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc011c80 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2634055d10 con 0x7f264406a5a0 2023-07-06T21:39:36.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f0a805ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7806e5f0 0x7f0a780b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f0a805ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a780b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f0a805ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a781608e0 0x7f0a7806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f0a805ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a780b4750 con 0x7f0a781608e0 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f0a805ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78162170 con 0x7f0a7806e5f0 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f0a805ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a78162450 con 0x7f0a780c52f0 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7e39b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7806e5f0 0x7f0a780b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7db9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a780b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7e39b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7806e5f0 0x7f0a780b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58798/0 (socket says 172.21.15.84:58798) 2023-07-06T21:39:36.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7e39b700 1 -- 172.21.15.84:0/3611795891 learned_addr learned my addr 172.21.15.84:0/3611795891 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc1e2270 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.988+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc0b1690 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7db9a700 1 -- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7806e5f0 msgr2=0x7f0a780b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7db9a700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7806e5f0 0x7f0a780b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7db9a700 1 -- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a781608e0 msgr2=0x7f0a7806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7db9a700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a781608e0 0x7f0a7806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f0a7db9a700 1 -- 172.21.15.84:0/3611795891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74009550 con 0x7f0a780c52f0 2023-07-06T21:39:36.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.989+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2fb8054950 con 0x7f2fc00b4370 2023-07-06T21:39:36.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.990+0000 7f0a7db9a700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a780b4090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e810 tx=0x7f0a6800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.990+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc0c3be0 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.991+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6800c070 con 0x7f0a780c52f0 2023-07-06T21:39:36.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.991+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a68013610 con 0x7f0a780c52f0 2023-07-06T21:39:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.991+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680139a0 con 0x7f0a780c52f0 2023-07-06T21:39:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.991+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a78162730 con 0x7f0a780c52f0 2023-07-06T21:39:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.991+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a781550a0 con 0x7f0a780c52f0 2023-07-06T21:39:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.992+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3dcc01c460 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.993+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a600052f0 con 0x7f0a780c52f0 2023-07-06T21:39:36.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.994+0000 7f3dd97fa700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3dc806adb0 0x7f3dc806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.994+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3dcc089040 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.995+0000 7f3ddbfff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3dc806adb0 0x7f3dc806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.996+0000 7f3ddbfff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3dc806adb0 0x7f3dc806d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc15bc90 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.996+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0a68013b00 con 0x7f0a780c52f0 2023-07-06T21:39:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.998+0000 7f0a6effd700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a5c06adb0 0x7f0a5c06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.998+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0a680893f0 con 0x7f0a780c52f0 2023-07-06T21:39:36.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:35.998+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3dcc0bb310 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.000+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0a680544a0 con 0x7f0a780c52f0 2023-07-06T21:39:36.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.016+0000 7f0a7e39b700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a5c06adb0 0x7f0a5c06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.020+0000 7f0a7e39b700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a5c06adb0 0x7f0a5c06d260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009fd0 tx=0x7f0a7401d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.040+0000 7f0bda3ff700 1 Processor -- start 2023-07-06T21:39:36.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.041+0000 7f0bda3ff700 1 -- start start 2023-07-06T21:39:36.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.043+0000 7f0bda3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0bd40cbcd0 0x7f0bd40adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.043+0000 7f0bda3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd415ccc0 0x7f0bd4069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.043+0000 7f0bda3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bd4159a60 con 0x7f0bd406a3d0 2023-07-06T21:39:36.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.043+0000 7f0bda3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bd406a850 con 0x7f0bd40cbcd0 2023-07-06T21:39:36.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.043+0000 7f0bda3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd4159780 con 0x7f0bd415ccc0 2023-07-06T21:39:36.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.044+0000 7f0bd3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0bd40cbcd0 0x7f0bd40adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.044+0000 7f0bd3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0bd40cbcd0 0x7f0bd40adce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51048/0 (socket says 172.21.15.84:51048) 2023-07-06T21:39:36.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.044+0000 7f0bd3fff700 1 -- 172.21.15.84:0/3467141762 learned_addr learned my addr 172.21.15.84:0/3467141762 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.044+0000 7f0bd899c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0bd406a3d0 0x7f0bd4158a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47848/0 (socket says 172.21.15.84:47848) 2023-07-06T21:39:36.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.045+0000 7f0bd3fff700 1 -- 172.21.15.84:0/3467141762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd415ccc0 msgr2=0x7f0bd4069e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-06T21:39:36.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.045+0000 7f0bd3fff700 1 --2- 172.21.15.84:0/3467141762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd415ccc0 0x7f0bd4069e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.045+0000 7f0bd3fff700 1 -- 172.21.15.84:0/3467141762 >> v1:172.21.15.84:6789/0 conn(0x7f0bd406a3d0 legacy=0x7f0bd4158a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:36.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.045+0000 7f0bd3fff700 1 -- 172.21.15.84:0/3467141762 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd415c1e0 con 0x7f0bd40cbcd0 2023-07-06T21:39:36.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.046+0000 7f0bd3fff700 1 --2- 172.21.15.84:0/3467141762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd40adce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098f0 tx=0x7f0bc4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ef05f75f2b89a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.046+0000 7f0bd2ffd700 1 -- 172.21.15.84:0/3467141762 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4004230 con 0x7f0bd40cbcd0 2023-07-06T21:39:36.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.046+0000 7f0bd2ffd700 1 -- 172.21.15.84:0/3467141762 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc4004390 con 0x7f0bd40cbcd0 2023-07-06T21:39:36.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.046+0000 7f0bd2ffd700 1 -- 172.21.15.84:0/3467141762 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40036a0 con 0x7f0bd40cbcd0 2023-07-06T21:39:36.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.047+0000 7f0bda3ff700 1 -- 172.21.15.84:0/3467141762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd40cbcd0 msgr2=0x7f0bd40adce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.047+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/3467141762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd40adce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40098f0 tx=0x7f0bc4009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:36.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7f0bda3ff700 1 -- 172.21.15.84:0/3467141762 shutdown_connections 2023-07-06T21:39:36.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/3467141762 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd40adce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/3467141762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd415ccc0 0x7f0bd4069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7f0bda3ff700 1 -- 172.21.15.84:0/3467141762 >> 172.21.15.84:0/3467141762 conn(0x7f0bd40cb1c0 msgr2=0x7f0bd40cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.049+0000 7f0bda3ff700 1 -- 172.21.15.84:0/3467141762 shutdown_connections 2023-07-06T21:39:36.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.049+0000 7f0bda3ff700 1 -- 172.21.15.84:0/3467141762 wait complete. 2023-07-06T21:39:36.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.045+0000 7ff8275ff700 1 Processor -- start 2023-07-06T21:39:36.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.046+0000 7ff8275ff700 1 -- start start 2023-07-06T21:39:36.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7ff8275ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820160590 0x7ff8200d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7ff8275ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff820164280 0x7ff820164680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7ff8275ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff820164bc0 con 0x7ff820164e90 2023-07-06T21:39:36.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7ff8275ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8200cfee0 con 0x7ff820164280 2023-07-06T21:39:36.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.048+0000 7ff8275ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200ce730 con 0x7ff820160590 2023-07-06T21:39:36.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.049+0000 7ff824b9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff820164e90 0x7ff8200cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47856/0 (socket says 172.21.15.84:47856) 2023-07-06T21:39:36.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.049+0000 7ff824b9a700 1 -- 172.21.15.84:0/3857536430 learned_addr learned my addr 172.21.15.84:0/3857536430 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:36.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff82539b700 1 --2- 172.21.15.84:0/3857536430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820160590 0x7ff8200d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff825b9c700 1 --2- 172.21.15.84:0/3857536430 >> v2:172.21.15.107:3300/0 conn(0x7ff820164280 0x7ff820164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff82539b700 1 -- 172.21.15.84:0/3857536430 >> v2:172.21.15.107:3300/0 conn(0x7ff820164280 msgr2=0x7ff820164680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:36.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff82539b700 1 --2- 172.21.15.84:0/3857536430 >> v2:172.21.15.107:3300/0 conn(0x7ff820164280 0x7ff820164680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff82539b700 1 -- 172.21.15.84:0/3857536430 >> v1:172.21.15.84:6789/0 conn(0x7ff820164e90 legacy=0x7ff8200cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff82539b700 1 -- 172.21.15.84:0/3857536430 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8200cea10 con 0x7ff820160590 2023-07-06T21:39:36.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.050+0000 7ff825b9c700 1 --2- 172.21.15.84:0/3857536430 >> v2:172.21.15.107:3300/0 conn(0x7ff820164280 0x7ff820164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:36.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.051+0000 7ff82539b700 1 --2- 172.21.15.84:0/3857536430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820160590 0x7ff8200d10f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff810009840 tx=0x7ff810009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfd16d923ddbd03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.051+0000 7f0bda3ff700 1 Processor -- start 2023-07-06T21:39:36.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.052+0000 7f0bda3ff700 1 -- start start 2023-07-06T21:39:36.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7f0bda3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd406a3d0 0x7f0bd40c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7f0bda3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7f0bda3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 0x7f0bd414a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7f0bda3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40c7fb0 con 0x7f0bd415ccc0 2023-07-06T21:39:36.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7f0bda3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd40c7cd0 con 0x7f0bd40cbcd0 2023-07-06T21:39:36.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7f0bda3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd41437c0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7ff817fff700 1 -- 172.21.15.84:0/3857536430 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810004230 con 0x7ff820160590 2023-07-06T21:39:36.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7ff817fff700 1 -- 172.21.15.84:0/3857536430 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810004390 con 0x7ff820160590 2023-07-06T21:39:36.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.053+0000 7ff817fff700 1 -- 172.21.15.84:0/3857536430 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100036a0 con 0x7ff820160590 2023-07-06T21:39:36.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7ff8275ff700 1 -- 172.21.15.84:0/3857536430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820160590 msgr2=0x7ff8200d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3857536430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820160590 0x7ff8200d10f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff810009840 tx=0x7ff810009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:36.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7f0bd899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 0x7f0bd414a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7f0bd899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 0x7f0bd414a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51064/0 (socket says 172.21.15.84:51064) 2023-07-06T21:39:36.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7f0bd899c700 1 -- 172.21.15.84:0/1943578466 learned_addr learned my addr 172.21.15.84:0/1943578466 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7f0bd37fe700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.054+0000 7f0bd3fff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd406a3d0 0x7f0bd40c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7ff8275ff700 1 -- 172.21.15.84:0/3857536430 shutdown_connections 2023-07-06T21:39:36.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3857536430 >> v2:172.21.15.107:3300/0 conn(0x7ff820164280 0x7ff820164680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3857536430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820160590 0x7ff8200d10f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7ff8275ff700 1 -- 172.21.15.84:0/3857536430 >> 172.21.15.84:0/3857536430 conn(0x7ff82006b350 msgr2=0x7ff82006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7f0bd3fff700 1 -- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bd40cbcd0 msgr2=0x7f0bd414a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7f0bd3fff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd414a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7f0bd3fff700 1 -- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 msgr2=0x7f0bd414a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7f0bd3fff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 0x7f0bd414a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7f0bd3fff700 1 -- 172.21.15.84:0/1943578466 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc4009550 con 0x7f0bd406a3d0 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7f0bd899c700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 0x7f0bd414a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.056+0000 7f0bd3fff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd406a3d0 0x7f0bd40c7480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4015000 tx=0x7f0bc4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.056+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400b660 con 0x7f0bd406a3d0 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.056+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc400b7e0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.056+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400bb90 con 0x7f0bd406a3d0 2023-07-06T21:39:36.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.057+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bd4143aa0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.058+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bd40c79c0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7ff8275ff700 1 -- 172.21.15.84:0/3857536430 shutdown_connections 2023-07-06T21:39:36.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.055+0000 7ff8275ff700 1 -- 172.21.15.84:0/3857536430 wait complete. 2023-07-06T21:39:36.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.058+0000 7ff8275ff700 1 Processor -- start 2023-07-06T21:39:36.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.059+0000 7feb90b9d700 1 Processor -- start 2023-07-06T21:39:36.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.058+0000 7ff8275ff700 1 -- start start 2023-07-06T21:39:36.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.060+0000 7ff8275ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 0x7ff820070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.060+0000 7ff8275ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820164280 0x7ff8201569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.060+0000 7ff8275ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820164e90 0x7ff82015a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.060+0000 7ff8275ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200c6e20 con 0x7ff820164280 2023-07-06T21:39:36.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.060+0000 7ff8275ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8200c67f0 con 0x7ff820160590 2023-07-06T21:39:36.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.060+0000 7ff8275ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82015e9e0 con 0x7ff820164e90 2023-07-06T21:39:36.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.061+0000 7ff82539b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 0x7ff820070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.061+0000 7ff825b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820164e90 0x7ff82015a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.061+0000 7ff82539b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 0x7ff820070d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:58832/0 (socket says 172.21.15.84:58832) 2023-07-06T21:39:36.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.061+0000 7ff82539b700 1 -- 172.21.15.84:0/3822501544 learned_addr learned my addr 172.21.15.84:0/3822501544 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.061+0000 7ff824b9a700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820164280 0x7ff8201569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.062+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0bc4022070 con 0x7f0bd406a3d0 2023-07-06T21:39:36.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.063+0000 7f0bd0ff9700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bc006adb0 0x7f0bc006d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.063+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0bc408b030 con 0x7f0bd406a3d0 2023-07-06T21:39:36.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.062+0000 7feb90b9d700 1 -- start start 2023-07-06T21:39:36.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7feb90b9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c158350 0x7feb8c0af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7feb90b9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb8c0cb640 0x7feb8c0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7feb90b9d700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8c0ada60 con 0x7feb8c0ae770 2023-07-06T21:39:36.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7feb90b9d700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8c0af6e0 con 0x7feb8c0cb640 2023-07-06T21:39:36.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7feb90b9d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0ae2e0 con 0x7feb8c158350 2023-07-06T21:39:36.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7ff82539b700 1 -- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820164e90 msgr2=0x7ff82015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7ff82539b700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820164e90 0x7ff82015a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7ff82539b700 1 -- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820164280 msgr2=0x7ff8201569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7ff82539b700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820164280 0x7ff8201569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7ff82539b700 1 -- 172.21.15.84:0/3822501544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100094a0 con 0x7ff820160590 2023-07-06T21:39:36.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.064+0000 7f0bd37fe700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bc006adb0 0x7f0bc006d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.065+0000 7ff82539b700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 0x7ff820070d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8200c6640 tx=0x7ff810004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.065+0000 7f0bd37fe700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bc006adb0 0x7f0bc006d260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fd0 tx=0x7f0bc8006ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.065+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bd4143fc0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.066+0000 7feb8bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb8c0cb640 0x7feb8c0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.066+0000 7feb8bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb8c0cb640 0x7feb8c0c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51086/0 (socket says 172.21.15.84:51086) 2023-07-06T21:39:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.066+0000 7feb8bfff700 1 -- 172.21.15.84:0/2142745001 learned_addr learned my addr 172.21.15.84:0/2142745001 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.067+0000 7feb8bfff700 1 -- 172.21.15.84:0/2142745001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c158350 msgr2=0x7feb8c0af1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.067+0000 7feb8bfff700 1 --2- 172.21.15.84:0/2142745001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c158350 0x7feb8c0af1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.067+0000 7feb8bfff700 1 -- 172.21.15.84:0/2142745001 >> v1:172.21.15.107:6789/0 conn(0x7feb8c0ae770 legacy=0x7feb8c0caf30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-06T21:39:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.067+0000 7feb8bfff700 1 -- 172.21.15.84:0/2142745001 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c0afc80 con 0x7feb8c0cb640 2023-07-06T21:39:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810003d60 con 0x7ff820160590 2023-07-06T21:39:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff810011630 con 0x7ff820160590 2023-07-06T21:39:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100119e0 con 0x7ff820160590 2023-07-06T21:39:36.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8200c6ab0 con 0x7ff820160590 2023-07-06T21:39:36.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82015e420 con 0x7ff820160590 2023-07-06T21:39:36.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7feb8bfff700 1 --2- 172.21.15.84:0/2142745001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c0cb640 0x7feb8c0c8b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb84009730 tx=0x7feb84009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bbc3875355950f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7feb8a7fc700 1 -- 172.21.15.84:0/2142745001 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84003c20 con 0x7feb8c0cb640 2023-07-06T21:39:36.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7feb8a7fc700 1 -- 172.21.15.84:0/2142745001 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb84003d80 con 0x7feb8c0cb640 2023-07-06T21:39:36.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.068+0000 7feb8a7fc700 1 -- 172.21.15.84:0/2142745001 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8400c930 con 0x7feb8c0cb640 2023-07-06T21:39:36.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.069+0000 7feb90b9d700 1 -- 172.21.15.84:0/2142745001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c0cb640 msgr2=0x7feb8c0c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.069+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2142745001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c0cb640 0x7feb8c0c8b70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb84009730 tx=0x7feb84009760 comp rx=0 tx=0).stop 2023-07-06T21:39:36.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7feb90b9d700 1 -- 172.21.15.84:0/2142745001 shutdown_connections 2023-07-06T21:39:36.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2142745001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c0cb640 0x7feb8c0c8b70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2142745001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c158350 0x7feb8c0af1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7feb90b9d700 1 -- 172.21.15.84:0/2142745001 >> 172.21.15.84:0/2142745001 conn(0x7feb8c0ce520 msgr2=0x7feb8c0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7feb90b9d700 1 -- 172.21.15.84:0/2142745001 shutdown_connections 2023-07-06T21:39:36.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7feb90b9d700 1 -- 172.21.15.84:0/2142745001 wait complete. 2023-07-06T21:39:36.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.070+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8200c6c70 con 0x7ff820160590 2023-07-06T21:39:36.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.071+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0bd4143fc0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.073+0000 7feb90b9d700 1 Processor -- start 2023-07-06T21:39:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.073+0000 7feb90b9d700 1 -- start start 2023-07-06T21:39:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7feb90b9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb8c0ae770 0x7feb8c0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7feb90b9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c0cb640 0x7feb8c0ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7feb90b9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c158350 0x7feb8c0ab670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7feb90b9d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0c4b80 con 0x7feb8c158350 2023-07-06T21:39:36.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7feb90b9d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0b3d90 con 0x7feb8c0ae770 2023-07-06T21:39:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7feb90b9d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c0b4070 con 0x7feb8c0cb640 2023-07-06T21:39:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.076+0000 7feb8affd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c0cb640 0x7feb8c0ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.076+0000 7feb8affd700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c0cb640 0x7feb8c0ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52244/0 (socket says 172.21.15.84:52244) 2023-07-06T21:39:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.076+0000 7feb8affd700 1 -- 172.21.15.84:0/2757212853 learned_addr learned my addr 172.21.15.84:0/2757212853 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.076+0000 7feb8b7fe700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb8c0ae770 0x7feb8c0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.075+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff810022070 con 0x7ff820160590 2023-07-06T21:39:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7ff815ffb700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff80c06af10 0x7ff80c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff81008b650 con 0x7ff820160590 2023-07-06T21:39:36.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.078+0000 7ff824b9a700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff80c06af10 0x7ff80c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.080+0000 7ff824b9a700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff80c06af10 0x7ff80c06d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0012e0 tx=0x7ff81c001270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7feb8bfff700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c158350 0x7feb8c0ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7feb8bfff700 1 -- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c0cb640 msgr2=0x7feb8c0ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7feb8bfff700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c0cb640 0x7feb8c0ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7feb8bfff700 1 -- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb8c0ae770 msgr2=0x7feb8c0c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7feb8bfff700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb8c0ae770 0x7feb8c0c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.077+0000 7feb8bfff700 1 -- 172.21.15.84:0/2757212853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8400c3d0 con 0x7feb8c158350 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.078+0000 7feb8bfff700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c158350 0x7feb8c0ab670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feb84016000 tx=0x7feb84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.078+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84004030 con 0x7feb8c158350 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.078+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb840041b0 con 0x7feb8c158350 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.078+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84004560 con 0x7feb8c158350 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.080+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c158b00 con 0x7feb8c158350 2023-07-06T21:39:36.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.080+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c0b3020 con 0x7feb8c158350 2023-07-06T21:39:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.083+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8c0b3020 con 0x7feb8c158350 2023-07-06T21:39:36.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.084+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff810056890 con 0x7ff820160590 2023-07-06T21:39:36.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.094+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb840046c0 con 0x7feb8c158350 2023-07-06T21:39:36.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.096+0000 7feb73fff700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7806adb0 0x7feb7806d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.096+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb8408ada0 con 0x7feb8c158350 2023-07-06T21:39:36.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.096+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb8408f3b0 con 0x7feb8c158350 2023-07-06T21:39:36.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.097+0000 7feb8b7fe700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7806adb0 0x7feb7806d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.098+0000 7feb8b7fe700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7806adb0 0x7feb7806d260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009960 tx=0x7feb7c0090e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.348+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f980055a0 con 0x7f7fbc0ae770 2023-07-06T21:39:36.356 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-07-06T21:39:36.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.349+0000 7f7fab7fe700 1 -- 172.21.15.84:0/1173120734 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7fac015070 con 0x7f7fbc0ae770 2023-07-06T21:39:36.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.353+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7fa406af10 msgr2=0x7f7fa406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.353+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7fa406af10 0x7f7fa406d3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400d690 tx=0x7f7fb4015000 comp rx=0 tx=0).stop 2023-07-06T21:39:36.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.353+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 msgr2=0x7f7fbc0b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.353+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0b1350 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fac00b450 tx=0x7f7fac00b810 comp rx=0 tx=0).stop 2023-07-06T21:39:36.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.354+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 shutdown_connections 2023-07-06T21:39:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.354+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7fa406af10 0x7f7fa406d3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.354+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7fbc158350 0x7f7fbc156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.354+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7fbc0ae770 0x7f7fbc0b1350 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.354+0000 7f7fbbfff700 1 --2- 172.21.15.84:0/1173120734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fbc0aebc0 0x7f7fbc1f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.354+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 >> 172.21.15.84:0/1173120734 conn(0x7f7fbc0ce520 msgr2=0x7f7fbc06d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.355+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 shutdown_connections 2023-07-06T21:39:36.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.355+0000 7f7fbbfff700 1 -- 172.21.15.84:0/1173120734 wait complete. 2023-07-06T21:39:36.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-07-06T21:39:36.389 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.425+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2fc00c3820 con 0x7f2fc00b4370 2023-07-06T21:39:36.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.426+0000 7f2fb5ffb700 1 -- 172.21.15.84:0/3071385681 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2fb8057f70 con 0x7f2fc00b4370 2023-07-06T21:39:36.432 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2fa406adb0 msgr2=0x7f2fa406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2fa406adb0 0x7f2fa406d260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2fc006b720 tx=0x7f2fb0009000 comp rx=0 tx=0).stop 2023-07-06T21:39:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 msgr2=0x7f2fc0161180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 0x7f2fc0161180 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b780 tx=0x7f2fb800bb40 comp rx=0 tx=0).stop 2023-07-06T21:39:36.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 shutdown_connections 2023-07-06T21:39:36.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2fa406adb0 0x7f2fa406d260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2fc00b4370 0x7f2fc0161180 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2fc006a5a0 0x7f2fc00b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 --2- 172.21.15.84:0/3071385681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fc00ccc90 0x7f2fc00c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 >> 172.21.15.84:0/3071385681 conn(0x7f2fc00cb1c0 msgr2=0x7f2fc006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 shutdown_connections 2023-07-06T21:39:36.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.430+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3ddc0ca540 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.431+0000 7f2fc75ff700 1 -- 172.21.15.84:0/3071385681 wait complete. 2023-07-06T21:39:36.437 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:36.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.431+0000 7f3dd97fa700 1 -- 172.21.15.84:0/3310604901 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3dcc054160 con 0x7f3ddc06a5a0 2023-07-06T21:39:36.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.434+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3dc806adb0 msgr2=0x7f3dc806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.434+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3dc806adb0 0x7f3dc806d260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc15bc90 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.434+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ddc06a5a0 msgr2=0x7f3ddc0af4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.434+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ddc06a5a0 0x7f3ddc0af4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c400 tx=0x7f3dcc00c710 comp rx=0 tx=0).stop 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 shutdown_connections 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3dc806adb0 0x7f3dc806d260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ddc15ccc0 0x7f3ddc158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ddc06a5a0 0x7f3ddc0af4f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 --2- 172.21.15.84:0/3310604901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ddc0adce0 0x7f3ddc157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 >> 172.21.15.84:0/3310604901 conn(0x7f3ddc0cb1c0 msgr2=0x7f3ddc0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 shutdown_connections 2023-07-06T21:39:36.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.435+0000 7f3de31ff700 1 -- 172.21.15.84:0/3310604901 wait complete. 2023-07-06T21:39:36.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.444+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f26441464d0 con 0x7f264406a5a0 2023-07-06T21:39:36.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.445+0000 7f2640ff9700 1 -- 172.21.15.84:0/2414440646 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2634059330 con 0x7f264406a5a0 2023-07-06T21:39:36.448 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:36.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.448+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0bd4143fc0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.449+0000 7f0bd0ff9700 1 -- 172.21.15.84:0/1943578466 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0bd4143fc0 con 0x7f0bd406a3d0 2023-07-06T21:39:36.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.449+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f262406adb0 msgr2=0x7f262406d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.449+0000 7f26499ff700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f262406adb0 0x7f262406d260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2644163220 tx=0x7f262c009040 comp rx=0 tx=0).stop 2023-07-06T21:39:36.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.449+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 msgr2=0x7f2644141d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.449+0000 7f26499ff700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 0x7f2644141d50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f263400b730 tx=0x7f2634004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.454 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:36.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 shutdown_connections 2023-07-06T21:39:36.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f262406adb0 0x7f262406d260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2644160830 0x7f264406c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f264406a5a0 0x7f2644141d50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 --2- 172.21.15.84:0/2414440646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26440c3600 0x7f264406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 >> 172.21.15.84:0/2414440646 conn(0x7f26440cf3d0 msgr2=0x7f26440cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 shutdown_connections 2023-07-06T21:39:36.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.450+0000 7f26499ff700 1 -- 172.21.15.84:0/2414440646 wait complete. 2023-07-06T21:39:36.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.451+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0a60005870 con 0x7f0a780c52f0 2023-07-06T21:39:36.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.452+0000 7f0a6effd700 1 -- 172.21.15.84:0/3611795891 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a68017020 con 0x7f0a780c52f0 2023-07-06T21:39:36.456 INFO:teuthology.orchestra.run.smithi084.stdout:55834574856 2023-07-06T21:39:36.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.452+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bc006adb0 msgr2=0x7f0bc006d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.452+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bc006adb0 0x7f0bc006d260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8006fd0 tx=0x7f0bc8006ec0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.452+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd406a3d0 msgr2=0x7f0bd40c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.453+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd406a3d0 0x7f0bd40c7480 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4015000 tx=0x7f0bc4004750 comp rx=0 tx=0).stop 2023-07-06T21:39:36.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 shutdown_connections 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bc006adb0 0x7f0bc006d260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd415ccc0 0x7f0bd414a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0bd406a3d0 0x7f0bd40c7480 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 --2- 172.21.15.84:0/1943578466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0bd40cbcd0 0x7f0bd414a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 >> 172.21.15.84:0/1943578466 conn(0x7f0bd40cb1c0 msgr2=0x7f0bd41512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 shutdown_connections 2023-07-06T21:39:36.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.454+0000 7f0bda3ff700 1 -- 172.21.15.84:0/1943578466 wait complete. 2023-07-06T21:39:36.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.456+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a5c06adb0 msgr2=0x7f0a5c06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.456+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a5c06adb0 0x7f0a5c06d260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009fd0 tx=0x7f0a7401d040 comp rx=0 tx=0).stop 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.456+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 msgr2=0x7f0a780b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.456+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a780b4090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800e810 tx=0x7f0a6800eb20 comp rx=0 tx=0).stop 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 shutdown_connections 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a5c06adb0 0x7f0a5c06d260 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009fd0 tx=0x7f0a7401d040 comp rx=0 tx=0).stop 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a781608e0 0x7f0a7806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7806e5f0 0x7f0a780b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 --2- 172.21.15.84:0/3611795891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a780c52f0 0x7f0a780b4090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 >> 172.21.15.84:0/3611795891 conn(0x7f0a780cb1c0 msgr2=0x7f0a7815fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.457+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 shutdown_connections 2023-07-06T21:39:36.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.458+0000 7f0a805ff700 1 -- 172.21.15.84:0/3611795891 wait complete. 2023-07-06T21:39:36.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-07-06T21:39:36.466 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-07-06T21:39:36.474 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-07-06T21:39:36.485 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2023-07-06T21:39:36.486 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-07-06T21:39:36.489 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.491+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff82015e700 con 0x7ff820160590 2023-07-06T21:39:36.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.492+0000 7ff815ffb700 1 -- 172.21.15.84:0/3822501544 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff810059eb0 con 0x7ff820160590 2023-07-06T21:39:36.495 INFO:teuthology.orchestra.run.smithi084.stdout:55834574855 2023-07-06T21:39:36.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff80c06af10 msgr2=0x7ff80c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff80c06af10 0x7ff80c06d3c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0012e0 tx=0x7ff81c001270 comp rx=0 tx=0).stop 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 msgr2=0x7ff820070d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 0x7ff820070d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8200c6640 tx=0x7ff810004750 comp rx=0 tx=0).stop 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 shutdown_connections 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff80c06af10 0x7ff80c06d3c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff820164e90 0x7ff82015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff820160590 0x7ff820070d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 --2- 172.21.15.84:0/3822501544 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff820164280 0x7ff8201569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 >> 172.21.15.84:0/3822501544 conn(0x7ff82006b350 msgr2=0x7ff8201622b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.494+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 shutdown_connections 2023-07-06T21:39:36.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.495+0000 7ff8275ff700 1 -- 172.21.15.84:0/3822501544 wait complete. 2023-07-06T21:39:36.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.501+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7feb8c158960 con 0x7feb8c158350 2023-07-06T21:39:36.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.502+0000 7feb73fff700 1 -- 172.21.15.84:0/2757212853 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb8c158960 con 0x7feb8c158350 2023-07-06T21:39:36.505 INFO:teuthology.orchestra.run.smithi084.stdout:55834574854 2023-07-06T21:39:36.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7806adb0 msgr2=0x7feb7806d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7806adb0 0x7feb7806d260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009960 tx=0x7feb7c0090e0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c158350 msgr2=0x7feb8c0ab670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c158350 0x7feb8c0ab670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feb84016000 tx=0x7feb84004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 shutdown_connections 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7806adb0 0x7feb7806d260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb8c158350 0x7feb8c0ab670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb8c0ae770 0x7feb8c0c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 --2- 172.21.15.84:0/2757212853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb8c0cb640 0x7feb8c0ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 >> 172.21.15.84:0/2757212853 conn(0x7feb8c0ce520 msgr2=0x7feb8c06d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 shutdown_connections 2023-07-06T21:39:36.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.504+0000 7feb90b9d700 1 -- 172.21.15.84:0/2757212853 wait complete. 2023-07-06T21:39:36.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-07-06T21:39:36.516 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-07-06T21:39:36.526 DEBUG:teuthology.parallel:result is None 2023-07-06T21:39:36.526 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T21:39:36.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:39:36.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.756+0000 7fb3ac9ff700 1 Processor -- start 2023-07-06T21:39:36.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.756+0000 7fb3ac9ff700 1 -- start start 2023-07-06T21:39:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3ac9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3a80b83c0 0x7fb3a80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3ac9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb3a80d3710 0x7fb3a80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3ac9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3ac9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a80cff10 con 0x7fb3a80b83c0 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3ac9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a80d0a80 con 0x7fb3a80d3710 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3ac9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b8790 con 0x7fb3a80cfa90 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3a659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3a80b83c0 0x7fb3a80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3a659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb3a80b83c0 0x7fb3a80cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51116/0 (socket says 172.21.15.84:51116) 2023-07-06T21:39:36.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3a659c700 1 -- 172.21.15.84:0/2217566640 learned_addr learned my addr 172.21.15.84:0/2217566640 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3a5d9b700 1 --2- 172.21.15.84:0/2217566640 >> v2:172.21.15.107:3300/0 conn(0x7fb3a80d3710 0x7fb3a80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.757+0000 7fb3a6d9d700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a659c700 1 -- 172.21.15.84:0/2217566640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 msgr2=0x7fb3a815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:36.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a659c700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a659c700 1 -- 172.21.15.84:0/2217566640 >> v2:172.21.15.107:3300/0 conn(0x7fb3a80d3710 msgr2=0x7fb3a80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:36.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a659c700 1 --2- 172.21.15.84:0/2217566640 >> v2:172.21.15.107:3300/0 conn(0x7fb3a80d3710 0x7fb3a80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a659c700 1 -- 172.21.15.84:0/2217566640 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80b4e70 con 0x7fb3a80b83c0 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a6d9d700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a659c700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80b83c0 0x7fb3a80cd200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80b8cb0 tx=0x7fb398009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f86f9798ee99cf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a559a700 1 -- 172.21.15.84:0/2217566640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398011830 con 0x7fb3a80b83c0 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a559a700 1 -- 172.21.15.84:0/2217566640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398011990 con 0x7fb3a80b83c0 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.758+0000 7fb3a559a700 1 -- 172.21.15.84:0/2217566640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398011d20 con 0x7fb3a80b83c0 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/2217566640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80b83c0 msgr2=0x7fb3a80cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80b83c0 0x7fb3a80cd200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80b8cb0 tx=0x7fb398009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/2217566640 shutdown_connections 2023-07-06T21:39:36.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/2217566640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80b83c0 0x7fb3a80cd200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/2217566640 >> v2:172.21.15.107:3300/0 conn(0x7fb3a80d3710 0x7fb3a80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/2217566640 >> 172.21.15.84:0/2217566640 conn(0x7fb3a806b350 msgr2=0x7fb3a806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/2217566640 shutdown_connections 2023-07-06T21:39:36.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.759+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/2217566640 wait complete. 2023-07-06T21:39:36.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.761+0000 7fb3ac9ff700 1 Processor -- start 2023-07-06T21:39:36.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.761+0000 7fb3ac9ff700 1 -- start start 2023-07-06T21:39:36.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3ac9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3a80b83c0 0x7fb3a8153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3ac9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a8156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3ac9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 0x7fb3a80b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3ac9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a8156b90 con 0x7fb3a80d3710 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3ac9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80c6810 con 0x7fb3a80b83c0 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3ac9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80c6af0 con 0x7fb3a80cfa90 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 0x7fb3a80b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 0x7fb3a80b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51122/0 (socket says 172.21.15.84:51122) 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 -- 172.21.15.84:0/1242701196 learned_addr learned my addr 172.21.15.84:0/1242701196 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a659c700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3a80b83c0 0x7fb3a8153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a5d9b700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a8156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 -- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 msgr2=0x7fb3a8156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a8156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 -- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3a80b83c0 msgr2=0x7fb3a8153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3a80b83c0 0x7fb3a8153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.762+0000 7fb3a6d9d700 1 -- 172.21.15.84:0/1242701196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb398009550 con 0x7fb3a80d3710 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.763+0000 7fb3a6d9d700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 0x7fb3a80b35a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0007e70 tx=0x7fb3a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.763+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000dd20 con 0x7fb3a80d3710 2023-07-06T21:39:36.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.763+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a000f040 con 0x7fb3a80d3710 2023-07-06T21:39:36.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.763+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a80c6dd0 con 0x7fb3a80d3710 2023-07-06T21:39:36.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.763+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000dea0 con 0x7fb3a80d3710 2023-07-06T21:39:36.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.763+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a81f2e30 con 0x7fb3a80d3710 2023-07-06T21:39:36.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.764+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a80c6670 con 0x7fb3a80d3710 2023-07-06T21:39:36.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.765+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb3a0003c60 con 0x7fb3a80d3710 2023-07-06T21:39:36.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.766+0000 7fb396ffd700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb38806ace0 0x7fb38806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:36.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.766+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb3a0089450 con 0x7fb3a80d3710 2023-07-06T21:39:36.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.766+0000 7fb3a659c700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb38806ace0 0x7fb38806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:36.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.767+0000 7fb3a659c700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb38806ace0 0x7fb38806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb398014040 tx=0x7fb398008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:36.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.771+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb3a80c6670 con 0x7fb3a80d3710 2023-07-06T21:39:36.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.964+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3a80b5430 con 0x7fb38806ace0 2023-07-06T21:39:36.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.972+0000 7fb396ffd700 1 -- 172.21.15.84:0/1242701196 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286603 (secure 0 0 0) 0x7fb398004040 con 0x7fb38806ace0 2023-07-06T21:39:36.979 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb38806ace0 msgr2=0x7fb38806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb38806ace0 0x7fb38806d190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb398014040 tx=0x7fb398008030 comp rx=0 tx=0).stop 2023-07-06T21:39:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 msgr2=0x7fb3a80b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 0x7fb3a80b35a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0007e70 tx=0x7fb3a000d3b0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 shutdown_connections 2023-07-06T21:39:36.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb38806ace0 0x7fb38806d190 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3a80d3710 0x7fb3a80b35a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3a80b83c0 0x7fb3a8153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 --2- 172.21.15.84:0/1242701196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3a80cfa90 0x7fb3a8156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 >> 172.21.15.84:0/1242701196 conn(0x7fb3a806b350 msgr2=0x7fb3a80d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 shutdown_connections 2023-07-06T21:39:36.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:36.977+0000 7fb3ac9ff700 1 -- 172.21.15.84:0/1242701196 wait complete. 2023-07-06T21:39:36.984 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:39:36.999 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-07-06T21:39:36.138673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.706323"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740251+0000","last_change":"2023-07-06T21:39:26.450219+0000","last_active":"2023-07-06T21:39:27.740251+0000","last_peered":"2023-07-06T21:39:27.740251+0000","last_clean":"2023-07-06T21:39:27.740251+0000","last_became_active":"2023-07-06T21:39:26.449158+0000","last_became_peered":"2023-07-06T21:39:26.449158+0000","last_unstale":"2023-07-06T21:39:27.740251+0000","last_undegraded":"2023-07-06T21:39:27.740251+0000","last_fullsized":"2023-07-06T21:39:27.740251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:05.032248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439046+0000","last_change":"2023-07-06T21:39:26.456425+0000","last_active":"2023-07-06T21:39:27.439046+0000","last_peered":"2023-07-06T21:39:27.439046+0000","last_clean":"2023-07-06T21:39:27.439046+0000","last_became_active":"2023-07-06T21:39:26.455881+0000","last_became_peered":"2023-07-06T21:39:26.455881+0000","last_unstale":"2023-07-06T21:39:27.439046+0000","last_undegraded":"2023-07-06T21:39:27.439046+0000","last_fullsized":"2023-07-06T21:39:27.439046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:29:06.029196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740972+0000","last_change":"2023-07-06T21:39:26.448957+0000","last_active":"2023-07-06T21:39:27.740972+0000","last_peered":"2023-07-06T21:39:27.740972+0000","last_clean":"2023-07-06T21:39:27.740972+0000","last_became_active":"2023-07-06T21:39:26.448647+0000","last_became_peered":"2023-07-06T21:39:26.448647+0000","last_unstale":"2023-07-06T21:39:27.740972+0000","last_undegraded":"2023-07-06T21:39:27.740972+0000","last_fullsized":"2023-07-06T21:39:27.740972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:55:48.758288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438699+0000","last_change":"2023-07-06T21:39:26.468418+0000","last_active":"2023-07-06T21:39:27.438699+0000","last_peered":"2023-07-06T21:39:27.438699+0000","last_clean":"2023-07-06T21:39:27.438699+0000","last_became_active":"2023-07-06T21:39:26.464060+0000","last_became_peered":"2023-07-06T21:39:26.464060+0000","last_unstale":"2023-07-06T21:39:27.438699+0000","last_undegraded":"2023-07-06T21:39:27.438699+0000","last_fullsized":"2023-07-06T21:39:27.438699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:07.305855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739951+0000","last_change":"2023-07-06T21:39:26.461152+0000","last_active":"2023-07-06T21:39:27.739951+0000","last_peered":"2023-07-06T21:39:27.739951+0000","last_clean":"2023-07-06T21:39:27.739951+0000","last_became_active":"2023-07-06T21:39:26.460854+0000","last_became_peered":"2023-07-06T21:39:26.460854+0000","last_unstale":"2023-07-06T21:39:27.739951+0000","last_undegraded":"2023-07-06T21:39:27.739951+0000","last_fullsized":"2023-07-06T21:39:27.739951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:48.843759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740102+0000","last_change":"2023-07-06T21:39:26.449460+0000","last_active":"2023-07-06T21:39:27.740102+0000","last_peered":"2023-07-06T21:39:27.740102+0000","last_clean":"2023-07-06T21:39:27.740102+0000","last_became_active":"2023-07-06T21:39:26.449145+0000","last_became_peered":"2023-07-06T21:39:26.449145+0000","last_unstale":"2023-07-06T21:39:27.740102+0000","last_undegraded":"2023-07-06T21:39:27.740102+0000","last_fullsized":"2023-07-06T21:39:27.740102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739621+0000","last_change":"2023-07-06T21:39:26.467462+0000","last_active":"2023-07-06T21:39:27.739621+0000","last_peered":"2023-07-06T21:39:27.739621+0000","last_clean":"2023-07-06T21:39:27.739621+0000","last_became_active":"2023-07-06T21:39:26.466827+0000","last_became_peered":"2023-07-06T21:39:26.466827+0000","last_unstale":"2023-07-06T21:39:27.739621+0000","last_undegraded":"2023-07-06T21:39:27.739621+0000","last_fullsized":"2023-07-06T21:39:27.739621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:15.799821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438400+0000","last_change":"2023-07-06T21:39:26.459017+0000","last_active":"2023-07-06T21:39:27.438400+0000","last_peered":"2023-07-06T21:39:27.438400+0000","last_clean":"2023-07-06T21:39:27.438400+0000","last_became_active":"2023-07-06T21:39:26.458749+0000","last_became_peered":"2023-07-06T21:39:26.458749+0000","last_unstale":"2023-07-06T21:39:27.438400+0000","last_undegraded":"2023-07-06T21:39:27.438400+0000","last_fullsized":"2023-07-06T21:39:27.438400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740615+0000","last_change":"2023-07-06T21:39:26.453377+0000","last_active":"2023-07-06T21:39:27.740615+0000","last_peered":"2023-07-06T21:39:27.740615+0000","last_clean":"2023-07-06T21:39:27.740615+0000","last_became_active":"2023-07-06T21:39:26.452864+0000","last_became_peered":"2023-07-06T21:39:26.452864+0000","last_unstale":"2023-07-06T21:39:27.740615+0000","last_undegraded":"2023-07-06T21:39:27.740615+0000","last_fullsized":"2023-07-06T21:39:27.740615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439184+0000","last_change":"2023-07-06T21:39:26.456191+0000","last_active":"2023-07-06T21:39:27.439184+0000","last_peered":"2023-07-06T21:39:27.439184+0000","last_clean":"2023-07-06T21:39:27.439184+0000","last_became_active":"2023-07-06T21:39:26.455459+0000","last_became_peered":"2023-07-06T21:39:26.455459+0000","last_unstale":"2023-07-06T21:39:27.439184+0000","last_undegraded":"2023-07-06T21:39:27.439184+0000","last_fullsized":"2023-07-06T21:39:27.439184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:11:05.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740952+0000","last_change":"2023-07-06T21:39:26.450168+0000","last_active":"2023-07-06T21:39:27.740952+0000","last_peered":"2023-07-06T21:39:27.740952+0000","last_clean":"2023-07-06T21:39:27.740952+0000","last_became_active":"2023-07-06T21:39:26.449224+0000","last_became_peered":"2023-07-06T21:39:26.449224+0000","last_unstale":"2023-07-06T21:39:27.740952+0000","last_undegraded":"2023-07-06T21:39:27.740952+0000","last_fullsized":"2023-07-06T21:39:27.740952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:26.926801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740041+0000","last_change":"2023-07-06T21:39:26.454525+0000","last_active":"2023-07-06T21:39:27.740041+0000","last_peered":"2023-07-06T21:39:27.740041+0000","last_clean":"2023-07-06T21:39:27.740041+0000","last_became_active":"2023-07-06T21:39:26.453872+0000","last_became_peered":"2023-07-06T21:39:26.453872+0000","last_unstale":"2023-07-06T21:39:27.740041+0000","last_undegraded":"2023-07-06T21:39:27.740041+0000","last_fullsized":"2023-07-06T21:39:27.740041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739959+0000","last_change":"2023-07-06T21:39:26.449670+0000","last_active":"2023-07-06T21:39:27.739959+0000","last_peered":"2023-07-06T21:39:27.739959+0000","last_clean":"2023-07-06T21:39:27.739959+0000","last_became_active":"2023-07-06T21:39:26.449116+0000","last_became_peered":"2023-07-06T21:39:26.449116+0000","last_unstale":"2023-07-06T21:39:27.739959+0000","last_undegraded":"2023-07-06T21:39:27.739959+0000","last_fullsized":"2023-07-06T21:39:27.739959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438835+0000","last_change":"2023-07-06T21:39:26.450117+0000","last_active":"2023-07-06T21:39:27.438835+0000","last_peered":"2023-07-06T21:39:27.438835+0000","last_clean":"2023-07-06T21:39:27.438835+0000","last_became_active":"2023-07-06T21:39:26.449800+0000","last_became_peered":"2023-07-06T21:39:26.449800+0000","last_unstale":"2023-07-06T21:39:27.438835+0000","last_undegraded":"2023-07-06T21:39:27.438835+0000","last_fullsized":"2023-07-06T21:39:27.438835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:15.799821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739809+0000","last_change":"2023-07-06T21:39:26.467633+0000","last_active":"2023-07-06T21:39:27.739809+0000","last_peered":"2023-07-06T21:39:27.739809+0000","last_clean":"2023-07-06T21:39:27.739809+0000","last_became_active":"2023-07-06T21:39:26.467194+0000","last_became_peered":"2023-07-06T21:39:26.467194+0000","last_unstale":"2023-07-06T21:39:27.739809+0000","last_undegraded":"2023-07-06T21:39:27.739809+0000","last_fullsized":"2023-07-06T21:39:27.739809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740052+0000","last_change":"2023-07-06T21:39:26.448432+0000","last_active":"2023-07-06T21:39:27.740052+0000","last_peered":"2023-07-06T21:39:27.740052+0000","last_clean":"2023-07-06T21:39:27.740052+0000","last_became_active":"2023-07-06T21:39:26.448091+0000","last_became_peered":"2023-07-06T21:39:26.448091+0000","last_unstale":"2023-07-06T21:39:27.740052+0000","last_undegraded":"2023-07-06T21:39:27.740052+0000","last_fullsized":"2023-07-06T21:39:27.740052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:36.282481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740808+0000","last_change":"2023-07-06T21:39:26.453156+0000","last_active":"2023-07-06T21:39:27.740808+0000","last_peered":"2023-07-06T21:39:27.740808+0000","last_clean":"2023-07-06T21:39:27.740808+0000","last_became_active":"2023-07-06T21:39:26.452499+0000","last_became_peered":"2023-07-06T21:39:26.452499+0000","last_unstale":"2023-07-06T21:39:27.740808+0000","last_undegraded":"2023-07-06T21:39:27.740808+0000","last_fullsized":"2023-07-06T21:39:27.740808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740102+0000","last_change":"2023-07-06T21:39:26.442470+0000","last_active":"2023-07-06T21:39:27.740102+0000","last_peered":"2023-07-06T21:39:27.740102+0000","last_clean":"2023-07-06T21:39:27.740102+0000","last_became_active":"2023-07-06T21:39:26.442162+0000","last_became_peered":"2023-07-06T21:39:26.442162+0000","last_unstale":"2023-07-06T21:39:27.740102+0000","last_undegraded":"2023-07-06T21:39:27.740102+0000","last_fullsized":"2023-07-06T21:39:27.740102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739929+0000","last_change":"2023-07-06T21:39:26.469782+0000","last_active":"2023-07-06T21:39:27.739929+0000","last_peered":"2023-07-06T21:39:27.739929+0000","last_clean":"2023-07-06T21:39:27.739929+0000","last_became_active":"2023-07-06T21:39:26.469492+0000","last_became_peered":"2023-07-06T21:39:26.469492+0000","last_unstale":"2023-07-06T21:39:27.739929+0000","last_undegraded":"2023-07-06T21:39:27.739929+0000","last_fullsized":"2023-07-06T21:39:27.739929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740637+0000","last_change":"2023-07-06T21:39:26.465478+0000","last_active":"2023-07-06T21:39:27.740637+0000","last_peered":"2023-07-06T21:39:27.740637+0000","last_clean":"2023-07-06T21:39:27.740637+0000","last_became_active":"2023-07-06T21:39:26.465171+0000","last_became_peered":"2023-07-06T21:39:26.465171+0000","last_unstale":"2023-07-06T21:39:27.740637+0000","last_undegraded":"2023-07-06T21:39:27.740637+0000","last_fullsized":"2023-07-06T21:39:27.740637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:29:06.029196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740290+0000","last_change":"2023-07-06T21:39:26.465956+0000","last_active":"2023-07-06T21:39:27.740290+0000","last_peered":"2023-07-06T21:39:27.740290+0000","last_clean":"2023-07-06T21:39:27.740290+0000","last_became_active":"2023-07-06T21:39:26.465672+0000","last_became_peered":"2023-07-06T21:39:26.465672+0000","last_unstale":"2023-07-06T21:39:27.740290+0000","last_undegraded":"2023-07-06T21:39:27.740290+0000","last_fullsized":"2023-07-06T21:39:27.740290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:55:48.758288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739961+0000","last_change":"2023-07-06T21:39:26.453279+0000","last_active":"2023-07-06T21:39:27.739961+0000","last_peered":"2023-07-06T21:39:27.739961+0000","last_clean":"2023-07-06T21:39:27.739961+0000","last_became_active":"2023-07-06T21:39:26.452832+0000","last_became_peered":"2023-07-06T21:39:26.452832+0000","last_unstale":"2023-07-06T21:39:27.739961+0000","last_undegraded":"2023-07-06T21:39:27.739961+0000","last_fullsized":"2023-07-06T21:39:27.739961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:57.797206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437200+0000","last_change":"2023-07-06T21:39:26.450178+0000","last_active":"2023-07-06T21:39:27.437200+0000","last_peered":"2023-07-06T21:39:27.437200+0000","last_clean":"2023-07-06T21:39:27.437200+0000","last_became_active":"2023-07-06T21:39:26.449355+0000","last_became_peered":"2023-07-06T21:39:26.449355+0000","last_unstale":"2023-07-06T21:39:27.437200+0000","last_undegraded":"2023-07-06T21:39:27.437200+0000","last_fullsized":"2023-07-06T21:39:27.437200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739821+0000","last_change":"2023-07-06T21:39:26.465115+0000","last_active":"2023-07-06T21:39:27.739821+0000","last_peered":"2023-07-06T21:39:27.739821+0000","last_clean":"2023-07-06T21:39:27.739821+0000","last_became_active":"2023-07-06T21:39:26.464450+0000","last_became_peered":"2023-07-06T21:39:26.464450+0000","last_unstale":"2023-07-06T21:39:27.739821+0000","last_undegraded":"2023-07-06T21:39:27.739821+0000","last_fullsized":"2023-07-06T21:39:27.739821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:14.860196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740941+0000","last_change":"2023-07-06T21:39:26.464834+0000","last_active":"2023-07-06T21:39:27.740941+0000","last_peered":"2023-07-06T21:39:27.740941+0000","last_clean":"2023-07-06T21:39:27.740941+0000","last_became_active":"2023-07-06T21:39:26.464519+0000","last_became_peered":"2023-07-06T21:39:26.464519+0000","last_unstale":"2023-07-06T21:39:27.740941+0000","last_undegraded":"2023-07-06T21:39:27.740941+0000","last_fullsized":"2023-07-06T21:39:27.740941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740675+0000","last_change":"2023-07-06T21:39:26.451689+0000","last_active":"2023-07-06T21:39:27.740675+0000","last_peered":"2023-07-06T21:39:27.740675+0000","last_clean":"2023-07-06T21:39:27.740675+0000","last_became_active":"2023-07-06T21:39:26.451397+0000","last_became_peered":"2023-07-06T21:39:26.451397+0000","last_unstale":"2023-07-06T21:39:27.740675+0000","last_undegraded":"2023-07-06T21:39:27.740675+0000","last_fullsized":"2023-07-06T21:39:27.740675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438981+0000","last_change":"2023-07-06T21:39:26.466401+0000","last_active":"2023-07-06T21:39:27.438981+0000","last_peered":"2023-07-06T21:39:27.438981+0000","last_clean":"2023-07-06T21:39:27.438981+0000","last_became_active":"2023-07-06T21:39:26.465938+0000","last_became_peered":"2023-07-06T21:39:26.465938+0000","last_unstale":"2023-07-06T21:39:27.438981+0000","last_undegraded":"2023-07-06T21:39:27.438981+0000","last_fullsized":"2023-07-06T21:39:27.438981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740292+0000","last_change":"2023-07-06T21:39:25.954109+0000","last_active":"2023-07-06T21:39:27.740292+0000","last_peered":"2023-07-06T21:39:27.740292+0000","last_clean":"2023-07-06T21:39:27.740292+0000","last_became_active":"2023-07-06T21:39:25.953788+0000","last_became_peered":"2023-07-06T21:39:25.953788+0000","last_unstale":"2023-07-06T21:39:27.740292+0000","last_undegraded":"2023-07-06T21:39:27.740292+0000","last_fullsized":"2023-07-06T21:39:27.740292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:16.727712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437382+0000","last_change":"2023-07-06T21:39:26.455786+0000","last_active":"2023-07-06T21:39:27.437382+0000","last_peered":"2023-07-06T21:39:27.437382+0000","last_clean":"2023-07-06T21:39:27.437382+0000","last_became_active":"2023-07-06T21:39:26.455415+0000","last_became_peered":"2023-07-06T21:39:26.455415+0000","last_unstale":"2023-07-06T21:39:27.437382+0000","last_undegraded":"2023-07-06T21:39:27.437382+0000","last_fullsized":"2023-07-06T21:39:27.437382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740781+0000","last_change":"2023-07-06T21:39:25.952238+0000","last_active":"2023-07-06T21:39:27.740781+0000","last_peered":"2023-07-06T21:39:27.740781+0000","last_clean":"2023-07-06T21:39:27.740781+0000","last_became_active":"2023-07-06T21:39:25.951890+0000","last_became_peered":"2023-07-06T21:39:25.951890+0000","last_unstale":"2023-07-06T21:39:27.740781+0000","last_undegraded":"2023-07-06T21:39:27.740781+0000","last_fullsized":"2023-07-06T21:39:27.740781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:23.911793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741204+0000","last_change":"2023-07-06T21:39:26.463846+0000","last_active":"2023-07-06T21:39:27.741204+0000","last_peered":"2023-07-06T21:39:27.741204+0000","last_clean":"2023-07-06T21:39:27.741204+0000","last_became_active":"2023-07-06T21:39:26.463530+0000","last_became_peered":"2023-07-06T21:39:26.463530+0000","last_unstale":"2023-07-06T21:39:27.741204+0000","last_undegraded":"2023-07-06T21:39:27.741204+0000","last_fullsized":"2023-07-06T21:39:27.741204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437140+0000","last_change":"2023-07-06T21:39:25.958386+0000","last_active":"2023-07-06T21:39:27.437140+0000","last_peered":"2023-07-06T21:39:27.437140+0000","last_clean":"2023-07-06T21:39:27.437140+0000","last_became_active":"2023-07-06T21:39:25.958049+0000","last_became_peered":"2023-07-06T21:39:25.958049+0000","last_unstale":"2023-07-06T21:39:27.437140+0000","last_undegraded":"2023-07-06T21:39:27.437140+0000","last_fullsized":"2023-07-06T21:39:27.437140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:35.274536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438577+0000","last_change":"2023-07-06T21:39:26.461047+0000","last_active":"2023-07-06T21:39:27.438577+0000","last_peered":"2023-07-06T21:39:27.438577+0000","last_clean":"2023-07-06T21:39:27.438577+0000","last_became_active":"2023-07-06T21:39:26.460608+0000","last_became_peered":"2023-07-06T21:39:26.460608+0000","last_unstale":"2023-07-06T21:39:27.438577+0000","last_undegraded":"2023-07-06T21:39:27.438577+0000","last_fullsized":"2023-07-06T21:39:27.438577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:28.800563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740834+0000","last_change":"2023-07-06T21:39:25.953273+0000","last_active":"2023-07-06T21:39:27.740834+0000","last_peered":"2023-07-06T21:39:27.740834+0000","last_clean":"2023-07-06T21:39:27.740834+0000","last_became_active":"2023-07-06T21:39:25.947023+0000","last_became_peered":"2023-07-06T21:39:25.947023+0000","last_unstale":"2023-07-06T21:39:27.740834+0000","last_undegraded":"2023-07-06T21:39:27.740834+0000","last_fullsized":"2023-07-06T21:39:27.740834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:13.123105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437137+0000","last_change":"2023-07-06T21:39:26.453860+0000","last_active":"2023-07-06T21:39:27.437137+0000","last_peered":"2023-07-06T21:39:27.437137+0000","last_clean":"2023-07-06T21:39:27.437137+0000","last_became_active":"2023-07-06T21:39:26.453597+0000","last_became_peered":"2023-07-06T21:39:26.453597+0000","last_unstale":"2023-07-06T21:39:27.437137+0000","last_undegraded":"2023-07-06T21:39:27.437137+0000","last_fullsized":"2023-07-06T21:39:27.437137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:11.680113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439047+0000","last_change":"2023-07-06T21:39:25.949777+0000","last_active":"2023-07-06T21:39:27.439047+0000","last_peered":"2023-07-06T21:39:27.439047+0000","last_clean":"2023-07-06T21:39:27.439047+0000","last_became_active":"2023-07-06T21:39:25.949379+0000","last_became_peered":"2023-07-06T21:39:25.949379+0000","last_unstale":"2023-07-06T21:39:27.439047+0000","last_undegraded":"2023-07-06T21:39:27.439047+0000","last_fullsized":"2023-07-06T21:39:27.439047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.852251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437382+0000","last_change":"2023-07-06T21:39:26.463815+0000","last_active":"2023-07-06T21:39:27.437382+0000","last_peered":"2023-07-06T21:39:27.437382+0000","last_clean":"2023-07-06T21:39:27.437382+0000","last_became_active":"2023-07-06T21:39:26.463557+0000","last_became_peered":"2023-07-06T21:39:26.463557+0000","last_unstale":"2023-07-06T21:39:27.437382+0000","last_undegraded":"2023-07-06T21:39:27.437382+0000","last_fullsized":"2023-07-06T21:39:27.437382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:17.735656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740131+0000","last_change":"2023-07-06T21:39:25.963442+0000","last_active":"2023-07-06T21:39:27.740131+0000","last_peered":"2023-07-06T21:39:27.740131+0000","last_clean":"2023-07-06T21:39:27.740131+0000","last_became_active":"2023-07-06T21:39:25.963058+0000","last_became_peered":"2023-07-06T21:39:25.963058+0000","last_unstale":"2023-07-06T21:39:27.740131+0000","last_undegraded":"2023-07-06T21:39:27.740131+0000","last_fullsized":"2023-07-06T21:39:27.740131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:50:54.935056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740332+0000","last_change":"2023-07-06T21:39:26.464284+0000","last_active":"2023-07-06T21:39:27.740332+0000","last_peered":"2023-07-06T21:39:27.740332+0000","last_clean":"2023-07-06T21:39:27.740332+0000","last_became_active":"2023-07-06T21:39:26.463874+0000","last_became_peered":"2023-07-06T21:39:26.463874+0000","last_unstale":"2023-07-06T21:39:27.740332+0000","last_undegraded":"2023-07-06T21:39:27.740332+0000","last_fullsized":"2023-07-06T21:39:27.740332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740751+0000","last_change":"2023-07-06T21:39:25.950225+0000","last_active":"2023-07-06T21:39:27.740751+0000","last_peered":"2023-07-06T21:39:27.740751+0000","last_clean":"2023-07-06T21:39:27.740751+0000","last_became_active":"2023-07-06T21:39:25.949924+0000","last_became_peered":"2023-07-06T21:39:25.949924+0000","last_unstale":"2023-07-06T21:39:27.740751+0000","last_undegraded":"2023-07-06T21:39:27.740751+0000","last_fullsized":"2023-07-06T21:39:27.740751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:50:54.935056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740824+0000","last_change":"2023-07-06T21:39:26.463354+0000","last_active":"2023-07-06T21:39:27.740824+0000","last_peered":"2023-07-06T21:39:27.740824+0000","last_clean":"2023-07-06T21:39:27.740824+0000","last_became_active":"2023-07-06T21:39:26.463039+0000","last_became_peered":"2023-07-06T21:39:26.463039+0000","last_unstale":"2023-07-06T21:39:27.740824+0000","last_undegraded":"2023-07-06T21:39:27.740824+0000","last_fullsized":"2023-07-06T21:39:27.740824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:11:05.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740254+0000","last_change":"2023-07-06T21:39:25.951134+0000","last_active":"2023-07-06T21:39:27.740254+0000","last_peered":"2023-07-06T21:39:27.740254+0000","last_clean":"2023-07-06T21:39:27.740254+0000","last_became_active":"2023-07-06T21:39:25.950821+0000","last_became_peered":"2023-07-06T21:39:25.950821+0000","last_unstale":"2023-07-06T21:39:27.740254+0000","last_undegraded":"2023-07-06T21:39:27.740254+0000","last_fullsized":"2023-07-06T21:39:27.740254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:11:04.208829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437340+0000","last_change":"2023-07-06T21:39:26.455921+0000","last_active":"2023-07-06T21:39:27.437340+0000","last_peered":"2023-07-06T21:39:27.437340+0000","last_clean":"2023-07-06T21:39:27.437340+0000","last_became_active":"2023-07-06T21:39:26.455574+0000","last_became_peered":"2023-07-06T21:39:26.455574+0000","last_unstale":"2023-07-06T21:39:27.437340+0000","last_undegraded":"2023-07-06T21:39:27.437340+0000","last_fullsized":"2023-07-06T21:39:27.437340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:56.153818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739801+0000","last_change":"2023-07-06T21:39:25.949803+0000","last_active":"2023-07-06T21:39:27.739801+0000","last_peered":"2023-07-06T21:39:27.739801+0000","last_clean":"2023-07-06T21:39:27.739801+0000","last_became_active":"2023-07-06T21:39:25.949490+0000","last_became_peered":"2023-07-06T21:39:25.949490+0000","last_unstale":"2023-07-06T21:39:27.739801+0000","last_undegraded":"2023-07-06T21:39:27.739801+0000","last_fullsized":"2023-07-06T21:39:27.739801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439090+0000","last_change":"2023-07-06T21:39:26.466204+0000","last_active":"2023-07-06T21:39:27.439090+0000","last_peered":"2023-07-06T21:39:27.439090+0000","last_clean":"2023-07-06T21:39:27.439090+0000","last_became_active":"2023-07-06T21:39:26.465576+0000","last_became_peered":"2023-07-06T21:39:26.465576+0000","last_unstale":"2023-07-06T21:39:27.439090+0000","last_undegraded":"2023-07-06T21:39:27.439090+0000","last_fullsized":"2023-07-06T21:39:27.439090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:24.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739667+0000","last_change":"2023-07-06T21:39:25.952231+0000","last_active":"2023-07-06T21:39:27.739667+0000","last_peered":"2023-07-06T21:39:27.739667+0000","last_clean":"2023-07-06T21:39:27.739667+0000","last_became_active":"2023-07-06T21:39:25.951945+0000","last_became_peered":"2023-07-06T21:39:25.951945+0000","last_unstale":"2023-07-06T21:39:27.739667+0000","last_undegraded":"2023-07-06T21:39:27.739667+0000","last_fullsized":"2023-07-06T21:39:27.739667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:23.565957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740467+0000","last_change":"2023-07-06T21:39:26.460570+0000","last_active":"2023-07-06T21:39:27.740467+0000","last_peered":"2023-07-06T21:39:27.740467+0000","last_clean":"2023-07-06T21:39:27.740467+0000","last_became_active":"2023-07-06T21:39:26.460310+0000","last_became_peered":"2023-07-06T21:39:26.460310+0000","last_unstale":"2023-07-06T21:39:27.740467+0000","last_undegraded":"2023-07-06T21:39:27.740467+0000","last_fullsized":"2023-07-06T21:39:27.740467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:36.282481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438489+0000","last_change":"2023-07-06T21:39:25.953216+0000","last_active":"2023-07-06T21:39:27.438489+0000","last_peered":"2023-07-06T21:39:27.438489+0000","last_clean":"2023-07-06T21:39:27.438489+0000","last_became_active":"2023-07-06T21:39:25.952808+0000","last_became_peered":"2023-07-06T21:39:25.952808+0000","last_unstale":"2023-07-06T21:39:27.438489+0000","last_undegraded":"2023-07-06T21:39:27.438489+0000","last_fullsized":"2023-07-06T21:39:27.438489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437724+0000","last_change":"2023-07-06T21:39:26.463251+0000","last_active":"2023-07-06T21:39:27.437724+0000","last_peered":"2023-07-06T21:39:27.437724+0000","last_clean":"2023-07-06T21:39:27.437724+0000","last_became_active":"2023-07-06T21:39:26.463003+0000","last_became_peered":"2023-07-06T21:39:26.463003+0000","last_unstale":"2023-07-06T21:39:27.437724+0000","last_undegraded":"2023-07-06T21:39:27.437724+0000","last_fullsized":"2023-07-06T21:39:27.437724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:29:06.029196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740986+0000","last_change":"2023-07-06T21:39:25.950786+0000","last_active":"2023-07-06T21:39:27.740986+0000","last_peered":"2023-07-06T21:39:27.740986+0000","last_clean":"2023-07-06T21:39:27.740986+0000","last_became_active":"2023-07-06T21:39:25.950367+0000","last_became_peered":"2023-07-06T21:39:25.950367+0000","last_unstale":"2023-07-06T21:39:27.740986+0000","last_undegraded":"2023-07-06T21:39:27.740986+0000","last_fullsized":"2023-07-06T21:39:27.740986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:54:55.781158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438568+0000","last_change":"2023-07-06T21:39:26.465718+0000","last_active":"2023-07-06T21:39:27.438568+0000","last_peered":"2023-07-06T21:39:27.438568+0000","last_clean":"2023-07-06T21:39:27.438568+0000","last_became_active":"2023-07-06T21:39:26.465470+0000","last_became_peered":"2023-07-06T21:39:26.465470+0000","last_unstale":"2023-07-06T21:39:27.438568+0000","last_undegraded":"2023-07-06T21:39:27.438568+0000","last_fullsized":"2023-07-06T21:39:27.438568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:14.131050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740563+0000","last_change":"2023-07-06T21:39:25.950737+0000","last_active":"2023-07-06T21:39:27.740563+0000","last_peered":"2023-07-06T21:39:27.740563+0000","last_clean":"2023-07-06T21:39:27.740563+0000","last_became_active":"2023-07-06T21:39:25.950252+0000","last_became_peered":"2023-07-06T21:39:25.950252+0000","last_unstale":"2023-07-06T21:39:27.740563+0000","last_undegraded":"2023-07-06T21:39:27.740563+0000","last_fullsized":"2023-07-06T21:39:27.740563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:12:39.095924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739914+0000","last_change":"2023-07-06T21:39:26.449765+0000","last_active":"2023-07-06T21:39:27.739914+0000","last_peered":"2023-07-06T21:39:27.739914+0000","last_clean":"2023-07-06T21:39:27.739914+0000","last_became_active":"2023-07-06T21:39:26.449453+0000","last_became_peered":"2023-07-06T21:39:26.449453+0000","last_unstale":"2023-07-06T21:39:27.739914+0000","last_undegraded":"2023-07-06T21:39:27.739914+0000","last_fullsized":"2023-07-06T21:39:27.739914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:11.680113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740451+0000","last_change":"2023-07-06T21:39:25.951600+0000","last_active":"2023-07-06T21:39:27.740451+0000","last_peered":"2023-07-06T21:39:27.740451+0000","last_clean":"2023-07-06T21:39:27.740451+0000","last_became_active":"2023-07-06T21:39:25.951237+0000","last_became_peered":"2023-07-06T21:39:25.951237+0000","last_unstale":"2023-07-06T21:39:27.740451+0000","last_undegraded":"2023-07-06T21:39:27.740451+0000","last_fullsized":"2023-07-06T21:39:27.740451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:07:22.474973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438610+0000","last_change":"2023-07-06T21:39:26.466709+0000","last_active":"2023-07-06T21:39:27.438610+0000","last_peered":"2023-07-06T21:39:27.438610+0000","last_clean":"2023-07-06T21:39:27.438610+0000","last_became_active":"2023-07-06T21:39:26.463332+0000","last_became_peered":"2023-07-06T21:39:26.463332+0000","last_unstale":"2023-07-06T21:39:27.438610+0000","last_undegraded":"2023-07-06T21:39:27.438610+0000","last_fullsized":"2023-07-06T21:39:27.438610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:11.539304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438882+0000","last_change":"2023-07-06T21:39:25.953896+0000","last_active":"2023-07-06T21:39:27.438882+0000","last_peered":"2023-07-06T21:39:27.438882+0000","last_clean":"2023-07-06T21:39:27.438882+0000","last_became_active":"2023-07-06T21:39:25.953607+0000","last_became_peered":"2023-07-06T21:39:25.953607+0000","last_unstale":"2023-07-06T21:39:27.438882+0000","last_undegraded":"2023-07-06T21:39:27.438882+0000","last_fullsized":"2023-07-06T21:39:27.438882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:23.911793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739808+0000","last_change":"2023-07-06T21:39:26.453327+0000","last_active":"2023-07-06T21:39:27.739808+0000","last_peered":"2023-07-06T21:39:27.739808+0000","last_clean":"2023-07-06T21:39:27.739808+0000","last_became_active":"2023-07-06T21:39:26.453059+0000","last_became_peered":"2023-07-06T21:39:26.453059+0000","last_unstale":"2023-07-06T21:39:27.739808+0000","last_undegraded":"2023-07-06T21:39:27.739808+0000","last_fullsized":"2023-07-06T21:39:27.739808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:07.305855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437418+0000","last_change":"2023-07-06T21:39:25.955876+0000","last_active":"2023-07-06T21:39:27.437418+0000","last_peered":"2023-07-06T21:39:27.437418+0000","last_clean":"2023-07-06T21:39:27.437418+0000","last_became_active":"2023-07-06T21:39:25.955527+0000","last_became_peered":"2023-07-06T21:39:25.955527+0000","last_unstale":"2023-07-06T21:39:27.437418+0000","last_undegraded":"2023-07-06T21:39:27.437418+0000","last_fullsized":"2023-07-06T21:39:27.437418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:14.791877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740703+0000","last_change":"2023-07-06T21:39:26.452205+0000","last_active":"2023-07-06T21:39:27.740703+0000","last_peered":"2023-07-06T21:39:27.740703+0000","last_clean":"2023-07-06T21:39:27.740703+0000","last_became_active":"2023-07-06T21:39:26.451938+0000","last_became_peered":"2023-07-06T21:39:26.451938+0000","last_unstale":"2023-07-06T21:39:27.740703+0000","last_undegraded":"2023-07-06T21:39:27.740703+0000","last_fullsized":"2023-07-06T21:39:27.740703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:24.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438473+0000","last_change":"2023-07-06T21:39:25.954596+0000","last_active":"2023-07-06T21:39:27.438473+0000","last_peered":"2023-07-06T21:39:27.438473+0000","last_clean":"2023-07-06T21:39:27.438473+0000","last_became_active":"2023-07-06T21:39:25.954316+0000","last_became_peered":"2023-07-06T21:39:25.954316+0000","last_unstale":"2023-07-06T21:39:27.438473+0000","last_undegraded":"2023-07-06T21:39:27.438473+0000","last_fullsized":"2023-07-06T21:39:27.438473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:15.833771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741088+0000","last_change":"2023-07-06T21:39:26.446371+0000","last_active":"2023-07-06T21:39:27.741088+0000","last_peered":"2023-07-06T21:39:27.741088+0000","last_clean":"2023-07-06T21:39:27.741088+0000","last_became_active":"2023-07-06T21:39:26.445294+0000","last_became_peered":"2023-07-06T21:39:26.445294+0000","last_unstale":"2023-07-06T21:39:27.741088+0000","last_undegraded":"2023-07-06T21:39:27.741088+0000","last_fullsized":"2023-07-06T21:39:27.741088+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:07:23.482918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740252+0000","last_change":"2023-07-06T21:39:25.954319+0000","last_active":"2023-07-06T21:39:27.740252+0000","last_peered":"2023-07-06T21:39:27.740252+0000","last_clean":"2023-07-06T21:39:27.740252+0000","last_became_active":"2023-07-06T21:39:25.954089+0000","last_became_peered":"2023-07-06T21:39:25.954089+0000","last_unstale":"2023-07-06T21:39:27.740252+0000","last_undegraded":"2023-07-06T21:39:27.740252+0000","last_fullsized":"2023-07-06T21:39:27.740252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:55.145873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740517+0000","last_change":"2023-07-06T21:39:26.463246+0000","last_active":"2023-07-06T21:39:27.740517+0000","last_peered":"2023-07-06T21:39:27.740517+0000","last_clean":"2023-07-06T21:39:27.740517+0000","last_became_active":"2023-07-06T21:39:26.462881+0000","last_became_peered":"2023-07-06T21:39:26.462881+0000","last_unstale":"2023-07-06T21:39:27.740517+0000","last_undegraded":"2023-07-06T21:39:27.740517+0000","last_fullsized":"2023-07-06T21:39:27.740517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:24.919738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.436988+0000","last_change":"2023-07-06T21:39:25.962410+0000","last_active":"2023-07-06T21:39:27.436988+0000","last_peered":"2023-07-06T21:39:27.436988+0000","last_clean":"2023-07-06T21:39:27.436988+0000","last_became_active":"2023-07-06T21:39:25.962150+0000","last_became_peered":"2023-07-06T21:39:25.962150+0000","last_unstale":"2023-07-06T21:39:27.436988+0000","last_undegraded":"2023-07-06T21:39:27.436988+0000","last_fullsized":"2023-07-06T21:39:27.436988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:47.835814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741202+0000","last_change":"2023-07-06T21:39:26.449718+0000","last_active":"2023-07-06T21:39:27.741202+0000","last_peered":"2023-07-06T21:39:27.741202+0000","last_clean":"2023-07-06T21:39:27.741202+0000","last_became_active":"2023-07-06T21:39:26.449444+0000","last_became_peered":"2023-07-06T21:39:26.449444+0000","last_unstale":"2023-07-06T21:39:27.741202+0000","last_undegraded":"2023-07-06T21:39:27.741202+0000","last_fullsized":"2023-07-06T21:39:27.741202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:24.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740436+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:39:27.740436+0000","last_peered":"2023-07-06T21:39:27.740436+0000","last_clean":"2023-07-06T21:39:27.740436+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:39:27.740436+0000","last_undegraded":"2023-07-06T21:39:27.740436+0000","last_fullsized":"2023-07-06T21:39:27.740436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:07.704609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439168+0000","last_change":"2023-07-06T21:39:25.962785+0000","last_active":"2023-07-06T21:39:27.439168+0000","last_peered":"2023-07-06T21:39:27.439168+0000","last_clean":"2023-07-06T21:39:27.439168+0000","last_became_active":"2023-07-06T21:39:25.962456+0000","last_became_peered":"2023-07-06T21:39:25.962456+0000","last_unstale":"2023-07-06T21:39:27.439168+0000","last_undegraded":"2023-07-06T21:39:27.439168+0000","last_fullsized":"2023-07-06T21:39:27.439168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:50:54.935056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740463+0000","last_change":"2023-07-06T21:39:26.457017+0000","last_active":"2023-07-06T21:39:27.740463+0000","last_peered":"2023-07-06T21:39:27.740463+0000","last_clean":"2023-07-06T21:39:27.740463+0000","last_became_active":"2023-07-06T21:39:26.456767+0000","last_became_peered":"2023-07-06T21:39:26.456767+0000","last_unstale":"2023-07-06T21:39:27.740463+0000","last_undegraded":"2023-07-06T21:39:27.740463+0000","last_fullsized":"2023-07-06T21:39:27.740463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:56.153818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438825+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:39:27.438825+0000","last_peered":"2023-07-06T21:39:27.438825+0000","last_clean":"2023-07-06T21:39:27.438825+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:39:27.438825+0000","last_undegraded":"2023-07-06T21:39:27.438825+0000","last_fullsized":"2023-07-06T21:39:27.438825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:11.309216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740664+0000","last_change":"2023-07-06T21:39:25.955813+0000","last_active":"2023-07-06T21:39:27.740664+0000","last_peered":"2023-07-06T21:39:27.740664+0000","last_clean":"2023-07-06T21:39:27.740664+0000","last_became_active":"2023-07-06T21:39:25.955570+0000","last_became_peered":"2023-07-06T21:39:25.955570+0000","last_unstale":"2023-07-06T21:39:27.740664+0000","last_undegraded":"2023-07-06T21:39:27.740664+0000","last_fullsized":"2023-07-06T21:39:27.740664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.852251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740595+0000","last_change":"2023-07-06T21:39:26.458071+0000","last_active":"2023-07-06T21:39:27.740595+0000","last_peered":"2023-07-06T21:39:27.740595+0000","last_clean":"2023-07-06T21:39:27.740595+0000","last_became_active":"2023-07-06T21:39:26.457809+0000","last_became_peered":"2023-07-06T21:39:26.457809+0000","last_unstale":"2023-07-06T21:39:27.740595+0000","last_undegraded":"2023-07-06T21:39:27.740595+0000","last_fullsized":"2023-07-06T21:39:27.740595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:17.735656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739565+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:39:27.739565+0000","last_peered":"2023-07-06T21:39:27.739565+0000","last_clean":"2023-07-06T21:39:27.739565+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:39:27.739565+0000","last_undegraded":"2023-07-06T21:39:27.739565+0000","last_fullsized":"2023-07-06T21:39:27.739565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:12.312730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740253+0000","last_change":"2023-07-06T21:39:25.951522+0000","last_active":"2023-07-06T21:39:27.740253+0000","last_peered":"2023-07-06T21:39:27.740253+0000","last_clean":"2023-07-06T21:39:27.740253+0000","last_became_active":"2023-07-06T21:39:25.951221+0000","last_became_peered":"2023-07-06T21:39:25.951221+0000","last_unstale":"2023-07-06T21:39:27.740253+0000","last_undegraded":"2023-07-06T21:39:27.740253+0000","last_fullsized":"2023-07-06T21:39:27.740253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:35.274536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741026+0000","last_change":"2023-07-06T21:39:26.456883+0000","last_active":"2023-07-06T21:39:27.741026+0000","last_peered":"2023-07-06T21:39:27.741026+0000","last_clean":"2023-07-06T21:39:27.741026+0000","last_became_active":"2023-07-06T21:39:26.456380+0000","last_became_peered":"2023-07-06T21:39:26.456380+0000","last_unstale":"2023-07-06T21:39:27.741026+0000","last_undegraded":"2023-07-06T21:39:27.741026+0000","last_fullsized":"2023-07-06T21:39:27.741026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:14.860196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438695+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:39:27.438695+0000","last_peered":"2023-07-06T21:39:27.438695+0000","last_clean":"2023-07-06T21:39:27.438695+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:39:27.438695+0000","last_undegraded":"2023-07-06T21:39:27.438695+0000","last_fullsized":"2023-07-06T21:39:27.438695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:49.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739675+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:39:27.739675+0000","last_peered":"2023-07-06T21:39:27.739675+0000","last_clean":"2023-07-06T21:39:27.739675+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:39:27.739675+0000","last_undegraded":"2023-07-06T21:39:27.739675+0000","last_fullsized":"2023-07-06T21:39:27.739675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:11.340670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437268+0000","last_change":"2023-07-06T21:39:25.953072+0000","last_active":"2023-07-06T21:39:27.437268+0000","last_peered":"2023-07-06T21:39:27.437268+0000","last_clean":"2023-07-06T21:39:27.437268+0000","last_became_active":"2023-07-06T21:39:25.952723+0000","last_became_peered":"2023-07-06T21:39:25.952723+0000","last_unstale":"2023-07-06T21:39:27.437268+0000","last_undegraded":"2023-07-06T21:39:27.437268+0000","last_fullsized":"2023-07-06T21:39:27.437268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:53.825665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438345+0000","last_change":"2023-07-06T21:39:26.459006+0000","last_active":"2023-07-06T21:39:27.438345+0000","last_peered":"2023-07-06T21:39:27.438345+0000","last_clean":"2023-07-06T21:39:27.438345+0000","last_became_active":"2023-07-06T21:39:26.458537+0000","last_became_peered":"2023-07-06T21:39:26.458537+0000","last_unstale":"2023-07-06T21:39:27.438345+0000","last_undegraded":"2023-07-06T21:39:27.438345+0000","last_fullsized":"2023-07-06T21:39:27.438345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739820+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:39:27.739820+0000","last_peered":"2023-07-06T21:39:27.739820+0000","last_clean":"2023-07-06T21:39:27.739820+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:39:27.739820+0000","last_undegraded":"2023-07-06T21:39:27.739820+0000","last_fullsized":"2023-07-06T21:39:27.739820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:59.602755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739590+0000","last_change":"2023-07-06T21:39:25.951755+0000","last_active":"2023-07-06T21:39:27.739590+0000","last_peered":"2023-07-06T21:39:27.739590+0000","last_clean":"2023-07-06T21:39:27.739590+0000","last_became_active":"2023-07-06T21:39:25.951412+0000","last_became_peered":"2023-07-06T21:39:25.951412+0000","last_unstale":"2023-07-06T21:39:27.739590+0000","last_undegraded":"2023-07-06T21:39:27.739590+0000","last_fullsized":"2023-07-06T21:39:27.739590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741071+0000","last_change":"2023-07-06T21:39:26.455874+0000","last_active":"2023-07-06T21:39:27.741071+0000","last_peered":"2023-07-06T21:39:27.741071+0000","last_clean":"2023-07-06T21:39:27.741071+0000","last_became_active":"2023-07-06T21:39:26.455628+0000","last_became_peered":"2023-07-06T21:39:26.455628+0000","last_unstale":"2023-07-06T21:39:27.741071+0000","last_undegraded":"2023-07-06T21:39:27.741071+0000","last_fullsized":"2023-07-06T21:39:27.741071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:17.735656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740286+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:39:27.740286+0000","last_peered":"2023-07-06T21:39:27.740286+0000","last_clean":"2023-07-06T21:39:27.740286+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:39:27.740286+0000","last_undegraded":"2023-07-06T21:39:27.740286+0000","last_fullsized":"2023-07-06T21:39:27.740286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:20.500360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740715+0000","last_change":"2023-07-06T21:39:25.959332+0000","last_active":"2023-07-06T21:39:27.740715+0000","last_peered":"2023-07-06T21:39:27.740715+0000","last_clean":"2023-07-06T21:39:27.740715+0000","last_became_active":"2023-07-06T21:39:25.958983+0000","last_became_peered":"2023-07-06T21:39:25.958983+0000","last_unstale":"2023-07-06T21:39:27.740715+0000","last_undegraded":"2023-07-06T21:39:27.740715+0000","last_fullsized":"2023-07-06T21:39:27.740715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:14.791877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740002+0000","last_change":"2023-07-06T21:39:26.455969+0000","last_active":"2023-07-06T21:39:27.740002+0000","last_peered":"2023-07-06T21:39:27.740002+0000","last_clean":"2023-07-06T21:39:27.740002+0000","last_became_active":"2023-07-06T21:39:26.455716+0000","last_became_peered":"2023-07-06T21:39:26.455716+0000","last_unstale":"2023-07-06T21:39:27.740002+0000","last_undegraded":"2023-07-06T21:39:27.740002+0000","last_fullsized":"2023-07-06T21:39:27.740002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437645+0000","last_change":"2023-07-06T21:39:25.962443+0000","last_active":"2023-07-06T21:39:27.437645+0000","last_peered":"2023-07-06T21:39:27.437645+0000","last_clean":"2023-07-06T21:39:27.437645+0000","last_became_active":"2023-07-06T21:39:25.962123+0000","last_became_peered":"2023-07-06T21:39:25.962123+0000","last_unstale":"2023-07-06T21:39:27.437645+0000","last_undegraded":"2023-07-06T21:39:27.437645+0000","last_fullsized":"2023-07-06T21:39:27.437645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:12.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740998+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:39:27.740998+0000","last_peered":"2023-07-06T21:39:27.740998+0000","last_clean":"2023-07-06T21:39:27.740998+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:39:27.740998+0000","last_undegraded":"2023-07-06T21:39:27.740998+0000","last_fullsized":"2023-07-06T21:39:27.740998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:59.602755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.436960+0000","last_change":"2023-07-06T21:39:26.463271+0000","last_active":"2023-07-06T21:39:27.436960+0000","last_peered":"2023-07-06T21:39:27.436960+0000","last_clean":"2023-07-06T21:39:27.436960+0000","last_became_active":"2023-07-06T21:39:26.463082+0000","last_became_peered":"2023-07-06T21:39:26.463082+0000","last_unstale":"2023-07-06T21:39:27.436960+0000","last_undegraded":"2023-07-06T21:39:27.436960+0000","last_fullsized":"2023-07-06T21:39:27.436960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:07.305855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438426+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:39:27.438426+0000","last_peered":"2023-07-06T21:39:27.438426+0000","last_clean":"2023-07-06T21:39:27.438426+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:39:27.438426+0000","last_undegraded":"2023-07-06T21:39:27.438426+0000","last_fullsized":"2023-07-06T21:39:27.438426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:05:58.605808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741247+0000","last_change":"2023-07-06T21:39:25.946579+0000","last_active":"2023-07-06T21:39:27.741247+0000","last_peered":"2023-07-06T21:39:27.741247+0000","last_clean":"2023-07-06T21:39:27.741247+0000","last_became_active":"2023-07-06T21:39:25.945265+0000","last_became_peered":"2023-07-06T21:39:25.945265+0000","last_unstale":"2023-07-06T21:39:27.741247+0000","last_undegraded":"2023-07-06T21:39:27.741247+0000","last_fullsized":"2023-07-06T21:39:27.741247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:14.791877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740077+0000","last_change":"2023-07-06T21:39:26.458730+0000","last_active":"2023-07-06T21:39:27.740077+0000","last_peered":"2023-07-06T21:39:27.740077+0000","last_clean":"2023-07-06T21:39:27.740077+0000","last_became_active":"2023-07-06T21:39:26.458438+0000","last_became_peered":"2023-07-06T21:39:26.458438+0000","last_unstale":"2023-07-06T21:39:27.740077+0000","last_undegraded":"2023-07-06T21:39:27.740077+0000","last_fullsized":"2023-07-06T21:39:27.740077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740082+0000","last_change":"2023-07-06T21:39:25.948372+0000","last_active":"2023-07-06T21:39:27.740082+0000","last_peered":"2023-07-06T21:39:27.740082+0000","last_clean":"2023-07-06T21:39:27.740082+0000","last_became_active":"2023-07-06T21:39:25.948140+0000","last_became_peered":"2023-07-06T21:39:25.948140+0000","last_unstale":"2023-07-06T21:39:27.740082+0000","last_undegraded":"2023-07-06T21:39:27.740082+0000","last_fullsized":"2023-07-06T21:39:27.740082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:10.672168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739884+0000","last_change":"2023-07-06T21:39:26.454713+0000","last_active":"2023-07-06T21:39:27.739884+0000","last_peered":"2023-07-06T21:39:27.739884+0000","last_clean":"2023-07-06T21:39:27.739884+0000","last_became_active":"2023-07-06T21:39:26.454273+0000","last_became_peered":"2023-07-06T21:39:26.454273+0000","last_unstale":"2023-07-06T21:39:27.739884+0000","last_undegraded":"2023-07-06T21:39:27.739884+0000","last_fullsized":"2023-07-06T21:39:27.739884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438831+0000","last_change":"2023-07-06T21:39:25.951864+0000","last_active":"2023-07-06T21:39:27.438831+0000","last_peered":"2023-07-06T21:39:27.438831+0000","last_clean":"2023-07-06T21:39:27.438831+0000","last_became_active":"2023-07-06T21:39:25.951527+0000","last_became_peered":"2023-07-06T21:39:25.951527+0000","last_unstale":"2023-07-06T21:39:27.438831+0000","last_undegraded":"2023-07-06T21:39:27.438831+0000","last_fullsized":"2023-07-06T21:39:27.438831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:23.569276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740558+0000","last_change":"2023-07-06T21:39:26.450910+0000","last_active":"2023-07-06T21:39:27.740558+0000","last_peered":"2023-07-06T21:39:27.740558+0000","last_clean":"2023-07-06T21:39:27.740558+0000","last_became_active":"2023-07-06T21:39:26.450613+0000","last_became_peered":"2023-07-06T21:39:26.450613+0000","last_unstale":"2023-07-06T21:39:27.740558+0000","last_undegraded":"2023-07-06T21:39:27.740558+0000","last_fullsized":"2023-07-06T21:39:27.740558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439232+0000","last_change":"2023-07-06T21:39:25.945875+0000","last_active":"2023-07-06T21:39:27.439232+0000","last_peered":"2023-07-06T21:39:27.439232+0000","last_clean":"2023-07-06T21:39:27.439232+0000","last_became_active":"2023-07-06T21:39:25.945533+0000","last_became_peered":"2023-07-06T21:39:25.945533+0000","last_unstale":"2023-07-06T21:39:27.439232+0000","last_undegraded":"2023-07-06T21:39:27.439232+0000","last_fullsized":"2023-07-06T21:39:27.439232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:25.918856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740487+0000","last_change":"2023-07-06T21:39:26.447102+0000","last_active":"2023-07-06T21:39:27.740487+0000","last_peered":"2023-07-06T21:39:27.740487+0000","last_clean":"2023-07-06T21:39:27.740487+0000","last_became_active":"2023-07-06T21:39:26.446750+0000","last_became_peered":"2023-07-06T21:39:26.446750+0000","last_unstale":"2023-07-06T21:39:27.740487+0000","last_undegraded":"2023-07-06T21:39:27.740487+0000","last_fullsized":"2023-07-06T21:39:27.740487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:15.799821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437289+0000","last_change":"2023-07-06T21:39:25.952004+0000","last_active":"2023-07-06T21:39:27.437289+0000","last_peered":"2023-07-06T21:39:27.437289+0000","last_clean":"2023-07-06T21:39:27.437289+0000","last_became_active":"2023-07-06T21:39:25.951708+0000","last_became_peered":"2023-07-06T21:39:25.951708+0000","last_unstale":"2023-07-06T21:39:27.437289+0000","last_undegraded":"2023-07-06T21:39:27.437289+0000","last_fullsized":"2023-07-06T21:39:27.437289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:13.123105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437012+0000","last_change":"2023-07-06T21:39:26.450232+0000","last_active":"2023-07-06T21:39:27.437012+0000","last_peered":"2023-07-06T21:39:27.437012+0000","last_clean":"2023-07-06T21:39:27.437012+0000","last_became_active":"2023-07-06T21:39:26.449627+0000","last_became_peered":"2023-07-06T21:39:26.449627+0000","last_unstale":"2023-07-06T21:39:27.437012+0000","last_undegraded":"2023-07-06T21:39:27.437012+0000","last_fullsized":"2023-07-06T21:39:27.437012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740957+0000","last_change":"2023-07-06T21:39:25.945282+0000","last_active":"2023-07-06T21:39:27.740957+0000","last_peered":"2023-07-06T21:39:27.740957+0000","last_clean":"2023-07-06T21:39:27.740957+0000","last_became_active":"2023-07-06T21:39:25.944898+0000","last_became_peered":"2023-07-06T21:39:25.944898+0000","last_unstale":"2023-07-06T21:39:27.740957+0000","last_undegraded":"2023-07-06T21:39:27.740957+0000","last_fullsized":"2023-07-06T21:39:27.740957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:55.145873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739887+0000","last_change":"2023-07-06T21:39:26.470499+0000","last_active":"2023-07-06T21:39:27.739887+0000","last_peered":"2023-07-06T21:39:27.739887+0000","last_clean":"2023-07-06T21:39:27.739887+0000","last_became_active":"2023-07-06T21:39:26.470270+0000","last_became_peered":"2023-07-06T21:39:26.470270+0000","last_unstale":"2023-07-06T21:39:27.739887+0000","last_undegraded":"2023-07-06T21:39:27.739887+0000","last_fullsized":"2023-07-06T21:39:27.739887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438288+0000","last_change":"2023-07-06T21:39:26.469954+0000","last_active":"2023-07-06T21:39:27.438288+0000","last_peered":"2023-07-06T21:39:27.438288+0000","last_clean":"2023-07-06T21:39:27.438288+0000","last_became_active":"2023-07-06T21:39:26.469618+0000","last_became_peered":"2023-07-06T21:39:26.469618+0000","last_unstale":"2023-07-06T21:39:27.438288+0000","last_undegraded":"2023-07-06T21:39:27.438288+0000","last_fullsized":"2023-07-06T21:39:27.438288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437005+0000","last_change":"2023-07-06T21:39:26.450370+0000","last_active":"2023-07-06T21:39:27.437005+0000","last_peered":"2023-07-06T21:39:27.437005+0000","last_clean":"2023-07-06T21:39:27.437005+0000","last_became_active":"2023-07-06T21:39:26.449980+0000","last_became_peered":"2023-07-06T21:39:26.449980+0000","last_unstale":"2023-07-06T21:39:27.437005+0000","last_undegraded":"2023-07-06T21:39:27.437005+0000","last_fullsized":"2023-07-06T21:39:27.437005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:57.797206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740231+0000","last_change":"2023-07-06T21:39:26.469630+0000","last_active":"2023-07-06T21:39:27.740231+0000","last_peered":"2023-07-06T21:39:27.740231+0000","last_clean":"2023-07-06T21:39:27.740231+0000","last_became_active":"2023-07-06T21:39:26.469358+0000","last_became_peered":"2023-07-06T21:39:26.469358+0000","last_unstale":"2023-07-06T21:39:27.740231+0000","last_undegraded":"2023-07-06T21:39:27.740231+0000","last_fullsized":"2023-07-06T21:39:27.740231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:14.131050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437564+0000","last_change":"2023-07-06T21:39:26.466945+0000","last_active":"2023-07-06T21:39:27.437564+0000","last_peered":"2023-07-06T21:39:27.437564+0000","last_clean":"2023-07-06T21:39:27.437564+0000","last_became_active":"2023-07-06T21:39:26.466613+0000","last_became_peered":"2023-07-06T21:39:26.466613+0000","last_unstale":"2023-07-06T21:39:27.437564+0000","last_undegraded":"2023-07-06T21:39:27.437564+0000","last_fullsized":"2023-07-06T21:39:27.437564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:24.919738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740297+0000","last_change":"2023-07-06T21:39:26.466793+0000","last_active":"2023-07-06T21:39:27.740297+0000","last_peered":"2023-07-06T21:39:27.740297+0000","last_clean":"2023-07-06T21:39:27.740297+0000","last_became_active":"2023-07-06T21:39:26.466585+0000","last_became_peered":"2023-07-06T21:39:26.466585+0000","last_unstale":"2023-07-06T21:39:27.740297+0000","last_undegraded":"2023-07-06T21:39:27.740297+0000","last_fullsized":"2023-07-06T21:39:27.740297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7869999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.544}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.395}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:39:37.006 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:39:37.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.229+0000 7f09971ff700 1 Processor -- start 2023-07-06T21:39:37.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.229+0000 7f09971ff700 1 -- start start 2023-07-06T21:39:37.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.230+0000 7f09971ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09900b83c0 0x7f09900cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.230+0000 7f09971ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f09900cfc60 0x7f09900d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.230+0000 7f09971ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09900d0a80 con 0x7f09900d3710 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.230+0000 7f09971ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09900b8790 con 0x7f09900b83c0 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.230+0000 7f09971ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09900b4e70 con 0x7f09900cfc60 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f0994f9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09900b83c0 0x7f09900cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f098ffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f09900d3710 0x7f09900cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47858/0 (socket says 172.21.15.84:47858) 2023-07-06T21:39:37.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f098ffff700 1 -- 172.21.15.84:0/3988062902 learned_addr learned my addr 172.21.15.84:0/3988062902 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f099579c700 1 --2- 172.21.15.84:0/3988062902 >> v2:172.21.15.107:3300/0 conn(0x7f09900cfc60 0x7f09900d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f098f7fe700 1 -- 172.21.15.84:0/3988062902 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320642253 0 0) 0x7f09900d0a80 con 0x7f09900d3710 2023-07-06T21:39:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f098f7fe700 1 -- 172.21.15.84:0/3988062902 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09800035b0 con 0x7f09900d3710 2023-07-06T21:39:37.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f0994f9b700 1 -- 172.21.15.84:0/3988062902 >> v2:172.21.15.107:3300/0 conn(0x7f09900cfc60 msgr2=0x7f09900d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f0994f9b700 1 --2- 172.21.15.84:0/3988062902 >> v2:172.21.15.107:3300/0 conn(0x7f09900cfc60 0x7f09900d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f0994f9b700 1 -- 172.21.15.84:0/3988062902 >> v1:172.21.15.84:6789/0 conn(0x7f09900d3710 legacy=0x7f09900cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f0994f9b700 1 -- 172.21.15.84:0/3988062902 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09900b5150 con 0x7f09900b83c0 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.231+0000 7f0994f9b700 1 --2- 172.21.15.84:0/3988062902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900cd200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09900d0030 tx=0x7f0984009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46b29317d128d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f098f7fe700 1 -- 172.21.15.84:0/3988062902 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984011830 con 0x7f09900b83c0 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f098f7fe700 1 -- 172.21.15.84:0/3988062902 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0984011990 con 0x7f09900b83c0 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f098f7fe700 1 -- 172.21.15.84:0/3988062902 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984011d20 con 0x7f09900b83c0 2023-07-06T21:39:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 -- 172.21.15.84:0/3988062902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 msgr2=0x7f09900cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 --2- 172.21.15.84:0/3988062902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900cd200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f09900d0030 tx=0x7f0984009b20 comp rx=0 tx=0).stop 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 -- 172.21.15.84:0/3988062902 shutdown_connections 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 --2- 172.21.15.84:0/3988062902 >> v2:172.21.15.107:3300/0 conn(0x7f09900cfc60 0x7f09900d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 --2- 172.21.15.84:0/3988062902 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900cd200 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 -- 172.21.15.84:0/3988062902 >> 172.21.15.84:0/3988062902 conn(0x7f099006b350 msgr2=0x7f099006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 -- 172.21.15.84:0/3988062902 shutdown_connections 2023-07-06T21:39:37.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.232+0000 7f09971ff700 1 -- 172.21.15.84:0/3988062902 wait complete. 2023-07-06T21:39:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.234+0000 7f09971ff700 1 Processor -- start 2023-07-06T21:39:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.234+0000 7f09971ff700 1 -- start start 2023-07-06T21:39:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f09971ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f09971ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09900cfc60 0x7f09900a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f09971ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09900d3710 0x7f09901f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f09971ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09900aa4f0 con 0x7f09900b83c0 2023-07-06T21:39:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f09971ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099015c8f0 con 0x7f09900cfc60 2023-07-06T21:39:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f09971ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099015cbd0 con 0x7f09900d3710 2023-07-06T21:39:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f0994f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f0994f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51138/0 (socket says 172.21.15.84:51138) 2023-07-06T21:39:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f0994f9b700 1 -- 172.21.15.84:0/3673339429 learned_addr learned my addr 172.21.15.84:0/3673339429 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f098ffff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09900cfc60 0x7f09900a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.235+0000 7f099579c700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09900d3710 0x7f09901f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f0994f9b700 1 -- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09900d3710 msgr2=0x7f09901f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f0994f9b700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09900d3710 0x7f09901f1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f0994f9b700 1 -- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09900cfc60 msgr2=0x7f09900a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f0994f9b700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09900cfc60 0x7f09900a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f0994f9b700 1 -- 172.21.15.84:0/3673339429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840094a0 con 0x7f09900b83c0 2023-07-06T21:39:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f0994f9b700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900c95e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09840090b0 tx=0x7f0984004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09840038a0 con 0x7f09900b83c0 2023-07-06T21:39:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098401e070 con 0x7f09900b83c0 2023-07-06T21:39:37.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099015ceb0 con 0x7f09900b83c0 2023-07-06T21:39:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0984003c60 con 0x7f09900b83c0 2023-07-06T21:39:37.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.236+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09901f19a0 con 0x7f09900b83c0 2023-07-06T21:39:37.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.237+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099015b340 con 0x7f09900b83c0 2023-07-06T21:39:37.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.238+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f098402d450 con 0x7f09900b83c0 2023-07-06T21:39:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.239+0000 7f098d7fa700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f097406ace0 0x7f097406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.239+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f098408a4e0 con 0x7f09900b83c0 2023-07-06T21:39:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.240+0000 7f098ffff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f097406ace0 0x7f097406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.240+0000 7f098ffff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f097406ace0 0x7f097406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09900d2630 tx=0x7f097c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:37.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.243+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0984055580 con 0x7f09900b83c0 2023-07-06T21:39:37.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.433+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f099015ca60 con 0x7f097406ace0 2023-07-06T21:39:37.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.441+0000 7f098d7fa700 1 -- 172.21.15.84:0/3673339429 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286603 (secure 0 0 0) 0x7f099015ca60 con 0x7f097406ace0 2023-07-06T21:39:37.447 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:37.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f097406ace0 msgr2=0x7f097406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f097406ace0 0x7f097406d190 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f09900d2630 tx=0x7f097c009040 comp rx=0 tx=0).stop 2023-07-06T21:39:37.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 msgr2=0x7f09900c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900c95e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09840090b0 tx=0x7f0984004750 comp rx=0 tx=0).stop 2023-07-06T21:39:37.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 shutdown_connections 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f097406ace0 0x7f097406d190 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09900d3710 0x7f09901f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09900b83c0 0x7f09900c95e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 --2- 172.21.15.84:0/3673339429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09900cfc60 0x7f09900a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 >> 172.21.15.84:0/3673339429 conn(0x7f099006b350 msgr2=0x7f09900b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 shutdown_connections 2023-07-06T21:39:37.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.445+0000 7f09971ff700 1 -- 172.21.15.84:0/3673339429 wait complete. 2023-07-06T21:39:37.451 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:39:37.466 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-07-06T21:39:36.138673+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218724,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754755996,"statfs":{"total":773094113280,"available":772870139904,"internally_reserved":0,"allocated":3182592,"data_stored":1901233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.706323"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740251+0000","last_change":"2023-07-06T21:39:26.450219+0000","last_active":"2023-07-06T21:39:27.740251+0000","last_peered":"2023-07-06T21:39:27.740251+0000","last_clean":"2023-07-06T21:39:27.740251+0000","last_became_active":"2023-07-06T21:39:26.449158+0000","last_became_peered":"2023-07-06T21:39:26.449158+0000","last_unstale":"2023-07-06T21:39:27.740251+0000","last_undegraded":"2023-07-06T21:39:27.740251+0000","last_fullsized":"2023-07-06T21:39:27.740251+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:06:05.032248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439046+0000","last_change":"2023-07-06T21:39:26.456425+0000","last_active":"2023-07-06T21:39:27.439046+0000","last_peered":"2023-07-06T21:39:27.439046+0000","last_clean":"2023-07-06T21:39:27.439046+0000","last_became_active":"2023-07-06T21:39:26.455881+0000","last_became_peered":"2023-07-06T21:39:26.455881+0000","last_unstale":"2023-07-06T21:39:27.439046+0000","last_undegraded":"2023-07-06T21:39:27.439046+0000","last_fullsized":"2023-07-06T21:39:27.439046+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:29:06.029196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740972+0000","last_change":"2023-07-06T21:39:26.448957+0000","last_active":"2023-07-06T21:39:27.740972+0000","last_peered":"2023-07-06T21:39:27.740972+0000","last_clean":"2023-07-06T21:39:27.740972+0000","last_became_active":"2023-07-06T21:39:26.448647+0000","last_became_peered":"2023-07-06T21:39:26.448647+0000","last_unstale":"2023-07-06T21:39:27.740972+0000","last_undegraded":"2023-07-06T21:39:27.740972+0000","last_fullsized":"2023-07-06T21:39:27.740972+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:55:48.758288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438699+0000","last_change":"2023-07-06T21:39:26.468418+0000","last_active":"2023-07-06T21:39:27.438699+0000","last_peered":"2023-07-06T21:39:27.438699+0000","last_clean":"2023-07-06T21:39:27.438699+0000","last_became_active":"2023-07-06T21:39:26.464060+0000","last_became_peered":"2023-07-06T21:39:26.464060+0000","last_unstale":"2023-07-06T21:39:27.438699+0000","last_undegraded":"2023-07-06T21:39:27.438699+0000","last_fullsized":"2023-07-06T21:39:27.438699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:07.305855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739951+0000","last_change":"2023-07-06T21:39:26.461152+0000","last_active":"2023-07-06T21:39:27.739951+0000","last_peered":"2023-07-06T21:39:27.739951+0000","last_clean":"2023-07-06T21:39:27.739951+0000","last_became_active":"2023-07-06T21:39:26.460854+0000","last_became_peered":"2023-07-06T21:39:26.460854+0000","last_unstale":"2023-07-06T21:39:27.739951+0000","last_undegraded":"2023-07-06T21:39:27.739951+0000","last_fullsized":"2023-07-06T21:39:27.739951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:48.843759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740102+0000","last_change":"2023-07-06T21:39:26.449460+0000","last_active":"2023-07-06T21:39:27.740102+0000","last_peered":"2023-07-06T21:39:27.740102+0000","last_clean":"2023-07-06T21:39:27.740102+0000","last_became_active":"2023-07-06T21:39:26.449145+0000","last_became_peered":"2023-07-06T21:39:26.449145+0000","last_unstale":"2023-07-06T21:39:27.740102+0000","last_undegraded":"2023-07-06T21:39:27.740102+0000","last_fullsized":"2023-07-06T21:39:27.740102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739621+0000","last_change":"2023-07-06T21:39:26.467462+0000","last_active":"2023-07-06T21:39:27.739621+0000","last_peered":"2023-07-06T21:39:27.739621+0000","last_clean":"2023-07-06T21:39:27.739621+0000","last_became_active":"2023-07-06T21:39:26.466827+0000","last_became_peered":"2023-07-06T21:39:26.466827+0000","last_unstale":"2023-07-06T21:39:27.739621+0000","last_undegraded":"2023-07-06T21:39:27.739621+0000","last_fullsized":"2023-07-06T21:39:27.739621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:15.799821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438400+0000","last_change":"2023-07-06T21:39:26.459017+0000","last_active":"2023-07-06T21:39:27.438400+0000","last_peered":"2023-07-06T21:39:27.438400+0000","last_clean":"2023-07-06T21:39:27.438400+0000","last_became_active":"2023-07-06T21:39:26.458749+0000","last_became_peered":"2023-07-06T21:39:26.458749+0000","last_unstale":"2023-07-06T21:39:27.438400+0000","last_undegraded":"2023-07-06T21:39:27.438400+0000","last_fullsized":"2023-07-06T21:39:27.438400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740615+0000","last_change":"2023-07-06T21:39:26.453377+0000","last_active":"2023-07-06T21:39:27.740615+0000","last_peered":"2023-07-06T21:39:27.740615+0000","last_clean":"2023-07-06T21:39:27.740615+0000","last_became_active":"2023-07-06T21:39:26.452864+0000","last_became_peered":"2023-07-06T21:39:26.452864+0000","last_unstale":"2023-07-06T21:39:27.740615+0000","last_undegraded":"2023-07-06T21:39:27.740615+0000","last_fullsized":"2023-07-06T21:39:27.740615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439184+0000","last_change":"2023-07-06T21:39:26.456191+0000","last_active":"2023-07-06T21:39:27.439184+0000","last_peered":"2023-07-06T21:39:27.439184+0000","last_clean":"2023-07-06T21:39:27.439184+0000","last_became_active":"2023-07-06T21:39:26.455459+0000","last_became_peered":"2023-07-06T21:39:26.455459+0000","last_unstale":"2023-07-06T21:39:27.439184+0000","last_undegraded":"2023-07-06T21:39:27.439184+0000","last_fullsized":"2023-07-06T21:39:27.439184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:11:05.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740952+0000","last_change":"2023-07-06T21:39:26.450168+0000","last_active":"2023-07-06T21:39:27.740952+0000","last_peered":"2023-07-06T21:39:27.740952+0000","last_clean":"2023-07-06T21:39:27.740952+0000","last_became_active":"2023-07-06T21:39:26.449224+0000","last_became_peered":"2023-07-06T21:39:26.449224+0000","last_unstale":"2023-07-06T21:39:27.740952+0000","last_undegraded":"2023-07-06T21:39:27.740952+0000","last_fullsized":"2023-07-06T21:39:27.740952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:26.926801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740041+0000","last_change":"2023-07-06T21:39:26.454525+0000","last_active":"2023-07-06T21:39:27.740041+0000","last_peered":"2023-07-06T21:39:27.740041+0000","last_clean":"2023-07-06T21:39:27.740041+0000","last_became_active":"2023-07-06T21:39:26.453872+0000","last_became_peered":"2023-07-06T21:39:26.453872+0000","last_unstale":"2023-07-06T21:39:27.740041+0000","last_undegraded":"2023-07-06T21:39:27.740041+0000","last_fullsized":"2023-07-06T21:39:27.740041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739959+0000","last_change":"2023-07-06T21:39:26.449670+0000","last_active":"2023-07-06T21:39:27.739959+0000","last_peered":"2023-07-06T21:39:27.739959+0000","last_clean":"2023-07-06T21:39:27.739959+0000","last_became_active":"2023-07-06T21:39:26.449116+0000","last_became_peered":"2023-07-06T21:39:26.449116+0000","last_unstale":"2023-07-06T21:39:27.739959+0000","last_undegraded":"2023-07-06T21:39:27.739959+0000","last_fullsized":"2023-07-06T21:39:27.739959+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438835+0000","last_change":"2023-07-06T21:39:26.450117+0000","last_active":"2023-07-06T21:39:27.438835+0000","last_peered":"2023-07-06T21:39:27.438835+0000","last_clean":"2023-07-06T21:39:27.438835+0000","last_became_active":"2023-07-06T21:39:26.449800+0000","last_became_peered":"2023-07-06T21:39:26.449800+0000","last_unstale":"2023-07-06T21:39:27.438835+0000","last_undegraded":"2023-07-06T21:39:27.438835+0000","last_fullsized":"2023-07-06T21:39:27.438835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:15.799821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739809+0000","last_change":"2023-07-06T21:39:26.467633+0000","last_active":"2023-07-06T21:39:27.739809+0000","last_peered":"2023-07-06T21:39:27.739809+0000","last_clean":"2023-07-06T21:39:27.739809+0000","last_became_active":"2023-07-06T21:39:26.467194+0000","last_became_peered":"2023-07-06T21:39:26.467194+0000","last_unstale":"2023-07-06T21:39:27.739809+0000","last_undegraded":"2023-07-06T21:39:27.739809+0000","last_fullsized":"2023-07-06T21:39:27.739809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740052+0000","last_change":"2023-07-06T21:39:26.448432+0000","last_active":"2023-07-06T21:39:27.740052+0000","last_peered":"2023-07-06T21:39:27.740052+0000","last_clean":"2023-07-06T21:39:27.740052+0000","last_became_active":"2023-07-06T21:39:26.448091+0000","last_became_peered":"2023-07-06T21:39:26.448091+0000","last_unstale":"2023-07-06T21:39:27.740052+0000","last_undegraded":"2023-07-06T21:39:27.740052+0000","last_fullsized":"2023-07-06T21:39:27.740052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:36.282481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740808+0000","last_change":"2023-07-06T21:39:26.453156+0000","last_active":"2023-07-06T21:39:27.740808+0000","last_peered":"2023-07-06T21:39:27.740808+0000","last_clean":"2023-07-06T21:39:27.740808+0000","last_became_active":"2023-07-06T21:39:26.452499+0000","last_became_peered":"2023-07-06T21:39:26.452499+0000","last_unstale":"2023-07-06T21:39:27.740808+0000","last_undegraded":"2023-07-06T21:39:27.740808+0000","last_fullsized":"2023-07-06T21:39:27.740808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740102+0000","last_change":"2023-07-06T21:39:26.442470+0000","last_active":"2023-07-06T21:39:27.740102+0000","last_peered":"2023-07-06T21:39:27.740102+0000","last_clean":"2023-07-06T21:39:27.740102+0000","last_became_active":"2023-07-06T21:39:26.442162+0000","last_became_peered":"2023-07-06T21:39:26.442162+0000","last_unstale":"2023-07-06T21:39:27.740102+0000","last_undegraded":"2023-07-06T21:39:27.740102+0000","last_fullsized":"2023-07-06T21:39:27.740102+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739929+0000","last_change":"2023-07-06T21:39:26.469782+0000","last_active":"2023-07-06T21:39:27.739929+0000","last_peered":"2023-07-06T21:39:27.739929+0000","last_clean":"2023-07-06T21:39:27.739929+0000","last_became_active":"2023-07-06T21:39:26.469492+0000","last_became_peered":"2023-07-06T21:39:26.469492+0000","last_unstale":"2023-07-06T21:39:27.739929+0000","last_undegraded":"2023-07-06T21:39:27.739929+0000","last_fullsized":"2023-07-06T21:39:27.739929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:39.100972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740637+0000","last_change":"2023-07-06T21:39:26.465478+0000","last_active":"2023-07-06T21:39:27.740637+0000","last_peered":"2023-07-06T21:39:27.740637+0000","last_clean":"2023-07-06T21:39:27.740637+0000","last_became_active":"2023-07-06T21:39:26.465171+0000","last_became_peered":"2023-07-06T21:39:26.465171+0000","last_unstale":"2023-07-06T21:39:27.740637+0000","last_undegraded":"2023-07-06T21:39:27.740637+0000","last_fullsized":"2023-07-06T21:39:27.740637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:29:06.029196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740290+0000","last_change":"2023-07-06T21:39:26.465956+0000","last_active":"2023-07-06T21:39:27.740290+0000","last_peered":"2023-07-06T21:39:27.740290+0000","last_clean":"2023-07-06T21:39:27.740290+0000","last_became_active":"2023-07-06T21:39:26.465672+0000","last_became_peered":"2023-07-06T21:39:26.465672+0000","last_unstale":"2023-07-06T21:39:27.740290+0000","last_undegraded":"2023-07-06T21:39:27.740290+0000","last_fullsized":"2023-07-06T21:39:27.740290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:55:48.758288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739961+0000","last_change":"2023-07-06T21:39:26.453279+0000","last_active":"2023-07-06T21:39:27.739961+0000","last_peered":"2023-07-06T21:39:27.739961+0000","last_clean":"2023-07-06T21:39:27.739961+0000","last_became_active":"2023-07-06T21:39:26.452832+0000","last_became_peered":"2023-07-06T21:39:26.452832+0000","last_unstale":"2023-07-06T21:39:27.739961+0000","last_undegraded":"2023-07-06T21:39:27.739961+0000","last_fullsized":"2023-07-06T21:39:27.739961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:57.797206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437200+0000","last_change":"2023-07-06T21:39:26.450178+0000","last_active":"2023-07-06T21:39:27.437200+0000","last_peered":"2023-07-06T21:39:27.437200+0000","last_clean":"2023-07-06T21:39:27.437200+0000","last_became_active":"2023-07-06T21:39:26.449355+0000","last_became_peered":"2023-07-06T21:39:26.449355+0000","last_unstale":"2023-07-06T21:39:27.437200+0000","last_undegraded":"2023-07-06T21:39:27.437200+0000","last_fullsized":"2023-07-06T21:39:27.437200+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739821+0000","last_change":"2023-07-06T21:39:26.465115+0000","last_active":"2023-07-06T21:39:27.739821+0000","last_peered":"2023-07-06T21:39:27.739821+0000","last_clean":"2023-07-06T21:39:27.739821+0000","last_became_active":"2023-07-06T21:39:26.464450+0000","last_became_peered":"2023-07-06T21:39:26.464450+0000","last_unstale":"2023-07-06T21:39:27.739821+0000","last_undegraded":"2023-07-06T21:39:27.739821+0000","last_fullsized":"2023-07-06T21:39:27.739821+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:14.860196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740941+0000","last_change":"2023-07-06T21:39:26.464834+0000","last_active":"2023-07-06T21:39:27.740941+0000","last_peered":"2023-07-06T21:39:27.740941+0000","last_clean":"2023-07-06T21:39:27.740941+0000","last_became_active":"2023-07-06T21:39:26.464519+0000","last_became_peered":"2023-07-06T21:39:26.464519+0000","last_unstale":"2023-07-06T21:39:27.740941+0000","last_undegraded":"2023-07-06T21:39:27.740941+0000","last_fullsized":"2023-07-06T21:39:27.740941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740675+0000","last_change":"2023-07-06T21:39:26.451689+0000","last_active":"2023-07-06T21:39:27.740675+0000","last_peered":"2023-07-06T21:39:27.740675+0000","last_clean":"2023-07-06T21:39:27.740675+0000","last_became_active":"2023-07-06T21:39:26.451397+0000","last_became_peered":"2023-07-06T21:39:26.451397+0000","last_unstale":"2023-07-06T21:39:27.740675+0000","last_undegraded":"2023-07-06T21:39:27.740675+0000","last_fullsized":"2023-07-06T21:39:27.740675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438981+0000","last_change":"2023-07-06T21:39:26.466401+0000","last_active":"2023-07-06T21:39:27.438981+0000","last_peered":"2023-07-06T21:39:27.438981+0000","last_clean":"2023-07-06T21:39:27.438981+0000","last_became_active":"2023-07-06T21:39:26.465938+0000","last_became_peered":"2023-07-06T21:39:26.465938+0000","last_unstale":"2023-07-06T21:39:27.438981+0000","last_undegraded":"2023-07-06T21:39:27.438981+0000","last_fullsized":"2023-07-06T21:39:27.438981+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740292+0000","last_change":"2023-07-06T21:39:25.954109+0000","last_active":"2023-07-06T21:39:27.740292+0000","last_peered":"2023-07-06T21:39:27.740292+0000","last_clean":"2023-07-06T21:39:27.740292+0000","last_became_active":"2023-07-06T21:39:25.953788+0000","last_became_peered":"2023-07-06T21:39:25.953788+0000","last_unstale":"2023-07-06T21:39:27.740292+0000","last_undegraded":"2023-07-06T21:39:27.740292+0000","last_fullsized":"2023-07-06T21:39:27.740292+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:16.727712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437382+0000","last_change":"2023-07-06T21:39:26.455786+0000","last_active":"2023-07-06T21:39:27.437382+0000","last_peered":"2023-07-06T21:39:27.437382+0000","last_clean":"2023-07-06T21:39:27.437382+0000","last_became_active":"2023-07-06T21:39:26.455415+0000","last_became_peered":"2023-07-06T21:39:26.455415+0000","last_unstale":"2023-07-06T21:39:27.437382+0000","last_undegraded":"2023-07-06T21:39:27.437382+0000","last_fullsized":"2023-07-06T21:39:27.437382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740781+0000","last_change":"2023-07-06T21:39:25.952238+0000","last_active":"2023-07-06T21:39:27.740781+0000","last_peered":"2023-07-06T21:39:27.740781+0000","last_clean":"2023-07-06T21:39:27.740781+0000","last_became_active":"2023-07-06T21:39:25.951890+0000","last_became_peered":"2023-07-06T21:39:25.951890+0000","last_unstale":"2023-07-06T21:39:27.740781+0000","last_undegraded":"2023-07-06T21:39:27.740781+0000","last_fullsized":"2023-07-06T21:39:27.740781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:23.911793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741204+0000","last_change":"2023-07-06T21:39:26.463846+0000","last_active":"2023-07-06T21:39:27.741204+0000","last_peered":"2023-07-06T21:39:27.741204+0000","last_clean":"2023-07-06T21:39:27.741204+0000","last_became_active":"2023-07-06T21:39:26.463530+0000","last_became_peered":"2023-07-06T21:39:26.463530+0000","last_unstale":"2023-07-06T21:39:27.741204+0000","last_undegraded":"2023-07-06T21:39:27.741204+0000","last_fullsized":"2023-07-06T21:39:27.741204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:24.577221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437140+0000","last_change":"2023-07-06T21:39:25.958386+0000","last_active":"2023-07-06T21:39:27.437140+0000","last_peered":"2023-07-06T21:39:27.437140+0000","last_clean":"2023-07-06T21:39:27.437140+0000","last_became_active":"2023-07-06T21:39:25.958049+0000","last_became_peered":"2023-07-06T21:39:25.958049+0000","last_unstale":"2023-07-06T21:39:27.437140+0000","last_undegraded":"2023-07-06T21:39:27.437140+0000","last_fullsized":"2023-07-06T21:39:27.437140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:35.274536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438577+0000","last_change":"2023-07-06T21:39:26.461047+0000","last_active":"2023-07-06T21:39:27.438577+0000","last_peered":"2023-07-06T21:39:27.438577+0000","last_clean":"2023-07-06T21:39:27.438577+0000","last_became_active":"2023-07-06T21:39:26.460608+0000","last_became_peered":"2023-07-06T21:39:26.460608+0000","last_unstale":"2023-07-06T21:39:27.438577+0000","last_undegraded":"2023-07-06T21:39:27.438577+0000","last_fullsized":"2023-07-06T21:39:27.438577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:28.800563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740834+0000","last_change":"2023-07-06T21:39:25.953273+0000","last_active":"2023-07-06T21:39:27.740834+0000","last_peered":"2023-07-06T21:39:27.740834+0000","last_clean":"2023-07-06T21:39:27.740834+0000","last_became_active":"2023-07-06T21:39:25.947023+0000","last_became_peered":"2023-07-06T21:39:25.947023+0000","last_unstale":"2023-07-06T21:39:27.740834+0000","last_undegraded":"2023-07-06T21:39:27.740834+0000","last_fullsized":"2023-07-06T21:39:27.740834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:13.123105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437137+0000","last_change":"2023-07-06T21:39:26.453860+0000","last_active":"2023-07-06T21:39:27.437137+0000","last_peered":"2023-07-06T21:39:27.437137+0000","last_clean":"2023-07-06T21:39:27.437137+0000","last_became_active":"2023-07-06T21:39:26.453597+0000","last_became_peered":"2023-07-06T21:39:26.453597+0000","last_unstale":"2023-07-06T21:39:27.437137+0000","last_undegraded":"2023-07-06T21:39:27.437137+0000","last_fullsized":"2023-07-06T21:39:27.437137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:11.680113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439047+0000","last_change":"2023-07-06T21:39:25.949777+0000","last_active":"2023-07-06T21:39:27.439047+0000","last_peered":"2023-07-06T21:39:27.439047+0000","last_clean":"2023-07-06T21:39:27.439047+0000","last_became_active":"2023-07-06T21:39:25.949379+0000","last_became_peered":"2023-07-06T21:39:25.949379+0000","last_unstale":"2023-07-06T21:39:27.439047+0000","last_undegraded":"2023-07-06T21:39:27.439047+0000","last_fullsized":"2023-07-06T21:39:27.439047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.852251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437382+0000","last_change":"2023-07-06T21:39:26.463815+0000","last_active":"2023-07-06T21:39:27.437382+0000","last_peered":"2023-07-06T21:39:27.437382+0000","last_clean":"2023-07-06T21:39:27.437382+0000","last_became_active":"2023-07-06T21:39:26.463557+0000","last_became_peered":"2023-07-06T21:39:26.463557+0000","last_unstale":"2023-07-06T21:39:27.437382+0000","last_undegraded":"2023-07-06T21:39:27.437382+0000","last_fullsized":"2023-07-06T21:39:27.437382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:17.735656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740131+0000","last_change":"2023-07-06T21:39:25.963442+0000","last_active":"2023-07-06T21:39:27.740131+0000","last_peered":"2023-07-06T21:39:27.740131+0000","last_clean":"2023-07-06T21:39:27.740131+0000","last_became_active":"2023-07-06T21:39:25.963058+0000","last_became_peered":"2023-07-06T21:39:25.963058+0000","last_unstale":"2023-07-06T21:39:27.740131+0000","last_undegraded":"2023-07-06T21:39:27.740131+0000","last_fullsized":"2023-07-06T21:39:27.740131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:50:54.935056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740332+0000","last_change":"2023-07-06T21:39:26.464284+0000","last_active":"2023-07-06T21:39:27.740332+0000","last_peered":"2023-07-06T21:39:27.740332+0000","last_clean":"2023-07-06T21:39:27.740332+0000","last_became_active":"2023-07-06T21:39:26.463874+0000","last_became_peered":"2023-07-06T21:39:26.463874+0000","last_unstale":"2023-07-06T21:39:27.740332+0000","last_undegraded":"2023-07-06T21:39:27.740332+0000","last_fullsized":"2023-07-06T21:39:27.740332+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:54.833610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740751+0000","last_change":"2023-07-06T21:39:25.950225+0000","last_active":"2023-07-06T21:39:27.740751+0000","last_peered":"2023-07-06T21:39:27.740751+0000","last_clean":"2023-07-06T21:39:27.740751+0000","last_became_active":"2023-07-06T21:39:25.949924+0000","last_became_peered":"2023-07-06T21:39:25.949924+0000","last_unstale":"2023-07-06T21:39:27.740751+0000","last_undegraded":"2023-07-06T21:39:27.740751+0000","last_fullsized":"2023-07-06T21:39:27.740751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:50:54.935056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740824+0000","last_change":"2023-07-06T21:39:26.463354+0000","last_active":"2023-07-06T21:39:27.740824+0000","last_peered":"2023-07-06T21:39:27.740824+0000","last_clean":"2023-07-06T21:39:27.740824+0000","last_became_active":"2023-07-06T21:39:26.463039+0000","last_became_peered":"2023-07-06T21:39:26.463039+0000","last_unstale":"2023-07-06T21:39:27.740824+0000","last_undegraded":"2023-07-06T21:39:27.740824+0000","last_fullsized":"2023-07-06T21:39:27.740824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:11:05.216773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740254+0000","last_change":"2023-07-06T21:39:25.951134+0000","last_active":"2023-07-06T21:39:27.740254+0000","last_peered":"2023-07-06T21:39:27.740254+0000","last_clean":"2023-07-06T21:39:27.740254+0000","last_became_active":"2023-07-06T21:39:25.950821+0000","last_became_peered":"2023-07-06T21:39:25.950821+0000","last_unstale":"2023-07-06T21:39:27.740254+0000","last_undegraded":"2023-07-06T21:39:27.740254+0000","last_fullsized":"2023-07-06T21:39:27.740254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:11:04.208829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437340+0000","last_change":"2023-07-06T21:39:26.455921+0000","last_active":"2023-07-06T21:39:27.437340+0000","last_peered":"2023-07-06T21:39:27.437340+0000","last_clean":"2023-07-06T21:39:27.437340+0000","last_became_active":"2023-07-06T21:39:26.455574+0000","last_became_peered":"2023-07-06T21:39:26.455574+0000","last_unstale":"2023-07-06T21:39:27.437340+0000","last_undegraded":"2023-07-06T21:39:27.437340+0000","last_fullsized":"2023-07-06T21:39:27.437340+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:56.153818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739801+0000","last_change":"2023-07-06T21:39:25.949803+0000","last_active":"2023-07-06T21:39:27.739801+0000","last_peered":"2023-07-06T21:39:27.739801+0000","last_clean":"2023-07-06T21:39:27.739801+0000","last_became_active":"2023-07-06T21:39:25.949490+0000","last_became_peered":"2023-07-06T21:39:25.949490+0000","last_unstale":"2023-07-06T21:39:27.739801+0000","last_undegraded":"2023-07-06T21:39:27.739801+0000","last_fullsized":"2023-07-06T21:39:27.739801+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:10.531359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439090+0000","last_change":"2023-07-06T21:39:26.466204+0000","last_active":"2023-07-06T21:39:27.439090+0000","last_peered":"2023-07-06T21:39:27.439090+0000","last_clean":"2023-07-06T21:39:27.439090+0000","last_became_active":"2023-07-06T21:39:26.465576+0000","last_became_peered":"2023-07-06T21:39:26.465576+0000","last_unstale":"2023-07-06T21:39:27.439090+0000","last_undegraded":"2023-07-06T21:39:27.439090+0000","last_fullsized":"2023-07-06T21:39:27.439090+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:24.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739667+0000","last_change":"2023-07-06T21:39:25.952231+0000","last_active":"2023-07-06T21:39:27.739667+0000","last_peered":"2023-07-06T21:39:27.739667+0000","last_clean":"2023-07-06T21:39:27.739667+0000","last_became_active":"2023-07-06T21:39:25.951945+0000","last_became_peered":"2023-07-06T21:39:25.951945+0000","last_unstale":"2023-07-06T21:39:27.739667+0000","last_undegraded":"2023-07-06T21:39:27.739667+0000","last_fullsized":"2023-07-06T21:39:27.739667+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:23.565957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740467+0000","last_change":"2023-07-06T21:39:26.460570+0000","last_active":"2023-07-06T21:39:27.740467+0000","last_peered":"2023-07-06T21:39:27.740467+0000","last_clean":"2023-07-06T21:39:27.740467+0000","last_became_active":"2023-07-06T21:39:26.460310+0000","last_became_peered":"2023-07-06T21:39:26.460310+0000","last_unstale":"2023-07-06T21:39:27.740467+0000","last_undegraded":"2023-07-06T21:39:27.740467+0000","last_fullsized":"2023-07-06T21:39:27.740467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:36.282481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438489+0000","last_change":"2023-07-06T21:39:25.953216+0000","last_active":"2023-07-06T21:39:27.438489+0000","last_peered":"2023-07-06T21:39:27.438489+0000","last_clean":"2023-07-06T21:39:27.438489+0000","last_became_active":"2023-07-06T21:39:25.952808+0000","last_became_peered":"2023-07-06T21:39:25.952808+0000","last_unstale":"2023-07-06T21:39:27.438489+0000","last_undegraded":"2023-07-06T21:39:27.438489+0000","last_fullsized":"2023-07-06T21:39:27.438489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:33:07.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437724+0000","last_change":"2023-07-06T21:39:26.463251+0000","last_active":"2023-07-06T21:39:27.437724+0000","last_peered":"2023-07-06T21:39:27.437724+0000","last_clean":"2023-07-06T21:39:27.437724+0000","last_became_active":"2023-07-06T21:39:26.463003+0000","last_became_peered":"2023-07-06T21:39:26.463003+0000","last_unstale":"2023-07-06T21:39:27.437724+0000","last_undegraded":"2023-07-06T21:39:27.437724+0000","last_fullsized":"2023-07-06T21:39:27.437724+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:29:06.029196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740986+0000","last_change":"2023-07-06T21:39:25.950786+0000","last_active":"2023-07-06T21:39:27.740986+0000","last_peered":"2023-07-06T21:39:27.740986+0000","last_clean":"2023-07-06T21:39:27.740986+0000","last_became_active":"2023-07-06T21:39:25.950367+0000","last_became_peered":"2023-07-06T21:39:25.950367+0000","last_unstale":"2023-07-06T21:39:27.740986+0000","last_undegraded":"2023-07-06T21:39:27.740986+0000","last_fullsized":"2023-07-06T21:39:27.740986+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:54:55.781158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438568+0000","last_change":"2023-07-06T21:39:26.465718+0000","last_active":"2023-07-06T21:39:27.438568+0000","last_peered":"2023-07-06T21:39:27.438568+0000","last_clean":"2023-07-06T21:39:27.438568+0000","last_became_active":"2023-07-06T21:39:26.465470+0000","last_became_peered":"2023-07-06T21:39:26.465470+0000","last_unstale":"2023-07-06T21:39:27.438568+0000","last_undegraded":"2023-07-06T21:39:27.438568+0000","last_fullsized":"2023-07-06T21:39:27.438568+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:14.131050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740563+0000","last_change":"2023-07-06T21:39:25.950737+0000","last_active":"2023-07-06T21:39:27.740563+0000","last_peered":"2023-07-06T21:39:27.740563+0000","last_clean":"2023-07-06T21:39:27.740563+0000","last_became_active":"2023-07-06T21:39:25.950252+0000","last_became_peered":"2023-07-06T21:39:25.950252+0000","last_unstale":"2023-07-06T21:39:27.740563+0000","last_undegraded":"2023-07-06T21:39:27.740563+0000","last_fullsized":"2023-07-06T21:39:27.740563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:12:39.095924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739914+0000","last_change":"2023-07-06T21:39:26.449765+0000","last_active":"2023-07-06T21:39:27.739914+0000","last_peered":"2023-07-06T21:39:27.739914+0000","last_clean":"2023-07-06T21:39:27.739914+0000","last_became_active":"2023-07-06T21:39:26.449453+0000","last_became_peered":"2023-07-06T21:39:26.449453+0000","last_unstale":"2023-07-06T21:39:27.739914+0000","last_undegraded":"2023-07-06T21:39:27.739914+0000","last_fullsized":"2023-07-06T21:39:27.739914+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:11.680113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740451+0000","last_change":"2023-07-06T21:39:25.951600+0000","last_active":"2023-07-06T21:39:27.740451+0000","last_peered":"2023-07-06T21:39:27.740451+0000","last_clean":"2023-07-06T21:39:27.740451+0000","last_became_active":"2023-07-06T21:39:25.951237+0000","last_became_peered":"2023-07-06T21:39:25.951237+0000","last_unstale":"2023-07-06T21:39:27.740451+0000","last_undegraded":"2023-07-06T21:39:27.740451+0000","last_fullsized":"2023-07-06T21:39:27.740451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:07:22.474973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438610+0000","last_change":"2023-07-06T21:39:26.466709+0000","last_active":"2023-07-06T21:39:27.438610+0000","last_peered":"2023-07-06T21:39:27.438610+0000","last_clean":"2023-07-06T21:39:27.438610+0000","last_became_active":"2023-07-06T21:39:26.463332+0000","last_became_peered":"2023-07-06T21:39:26.463332+0000","last_unstale":"2023-07-06T21:39:27.438610+0000","last_undegraded":"2023-07-06T21:39:27.438610+0000","last_fullsized":"2023-07-06T21:39:27.438610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:11.539304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438882+0000","last_change":"2023-07-06T21:39:25.953896+0000","last_active":"2023-07-06T21:39:27.438882+0000","last_peered":"2023-07-06T21:39:27.438882+0000","last_clean":"2023-07-06T21:39:27.438882+0000","last_became_active":"2023-07-06T21:39:25.953607+0000","last_became_peered":"2023-07-06T21:39:25.953607+0000","last_unstale":"2023-07-06T21:39:27.438882+0000","last_undegraded":"2023-07-06T21:39:27.438882+0000","last_fullsized":"2023-07-06T21:39:27.438882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:23.911793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739808+0000","last_change":"2023-07-06T21:39:26.453327+0000","last_active":"2023-07-06T21:39:27.739808+0000","last_peered":"2023-07-06T21:39:27.739808+0000","last_clean":"2023-07-06T21:39:27.739808+0000","last_became_active":"2023-07-06T21:39:26.453059+0000","last_became_peered":"2023-07-06T21:39:26.453059+0000","last_unstale":"2023-07-06T21:39:27.739808+0000","last_undegraded":"2023-07-06T21:39:27.739808+0000","last_fullsized":"2023-07-06T21:39:27.739808+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:07.305855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437418+0000","last_change":"2023-07-06T21:39:25.955876+0000","last_active":"2023-07-06T21:39:27.437418+0000","last_peered":"2023-07-06T21:39:27.437418+0000","last_clean":"2023-07-06T21:39:27.437418+0000","last_became_active":"2023-07-06T21:39:25.955527+0000","last_became_peered":"2023-07-06T21:39:25.955527+0000","last_unstale":"2023-07-06T21:39:27.437418+0000","last_undegraded":"2023-07-06T21:39:27.437418+0000","last_fullsized":"2023-07-06T21:39:27.437418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:14.791877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740703+0000","last_change":"2023-07-06T21:39:26.452205+0000","last_active":"2023-07-06T21:39:27.740703+0000","last_peered":"2023-07-06T21:39:27.740703+0000","last_clean":"2023-07-06T21:39:27.740703+0000","last_became_active":"2023-07-06T21:39:26.451938+0000","last_became_peered":"2023-07-06T21:39:26.451938+0000","last_unstale":"2023-07-06T21:39:27.740703+0000","last_undegraded":"2023-07-06T21:39:27.740703+0000","last_fullsized":"2023-07-06T21:39:27.740703+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:24.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438473+0000","last_change":"2023-07-06T21:39:25.954596+0000","last_active":"2023-07-06T21:39:27.438473+0000","last_peered":"2023-07-06T21:39:27.438473+0000","last_clean":"2023-07-06T21:39:27.438473+0000","last_became_active":"2023-07-06T21:39:25.954316+0000","last_became_peered":"2023-07-06T21:39:25.954316+0000","last_unstale":"2023-07-06T21:39:27.438473+0000","last_undegraded":"2023-07-06T21:39:27.438473+0000","last_fullsized":"2023-07-06T21:39:27.438473+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:45:15.833771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741088+0000","last_change":"2023-07-06T21:39:26.446371+0000","last_active":"2023-07-06T21:39:27.741088+0000","last_peered":"2023-07-06T21:39:27.741088+0000","last_clean":"2023-07-06T21:39:27.741088+0000","last_became_active":"2023-07-06T21:39:26.445294+0000","last_became_peered":"2023-07-06T21:39:26.445294+0000","last_unstale":"2023-07-06T21:39:27.741088+0000","last_undegraded":"2023-07-06T21:39:27.741088+0000","last_fullsized":"2023-07-06T21:39:27.741088+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:07:23.482918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740252+0000","last_change":"2023-07-06T21:39:25.954319+0000","last_active":"2023-07-06T21:39:27.740252+0000","last_peered":"2023-07-06T21:39:27.740252+0000","last_clean":"2023-07-06T21:39:27.740252+0000","last_became_active":"2023-07-06T21:39:25.954089+0000","last_became_peered":"2023-07-06T21:39:25.954089+0000","last_unstale":"2023-07-06T21:39:27.740252+0000","last_undegraded":"2023-07-06T21:39:27.740252+0000","last_fullsized":"2023-07-06T21:39:27.740252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:55.145873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740517+0000","last_change":"2023-07-06T21:39:26.463246+0000","last_active":"2023-07-06T21:39:27.740517+0000","last_peered":"2023-07-06T21:39:27.740517+0000","last_clean":"2023-07-06T21:39:27.740517+0000","last_became_active":"2023-07-06T21:39:26.462881+0000","last_became_peered":"2023-07-06T21:39:26.462881+0000","last_unstale":"2023-07-06T21:39:27.740517+0000","last_undegraded":"2023-07-06T21:39:27.740517+0000","last_fullsized":"2023-07-06T21:39:27.740517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:24.919738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.436988+0000","last_change":"2023-07-06T21:39:25.962410+0000","last_active":"2023-07-06T21:39:27.436988+0000","last_peered":"2023-07-06T21:39:27.436988+0000","last_clean":"2023-07-06T21:39:27.436988+0000","last_became_active":"2023-07-06T21:39:25.962150+0000","last_became_peered":"2023-07-06T21:39:25.962150+0000","last_unstale":"2023-07-06T21:39:27.436988+0000","last_undegraded":"2023-07-06T21:39:27.436988+0000","last_fullsized":"2023-07-06T21:39:27.436988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:47.835814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741202+0000","last_change":"2023-07-06T21:39:26.449718+0000","last_active":"2023-07-06T21:39:27.741202+0000","last_peered":"2023-07-06T21:39:27.741202+0000","last_clean":"2023-07-06T21:39:27.741202+0000","last_became_active":"2023-07-06T21:39:26.449444+0000","last_became_peered":"2023-07-06T21:39:26.449444+0000","last_unstale":"2023-07-06T21:39:27.741202+0000","last_undegraded":"2023-07-06T21:39:27.741202+0000","last_fullsized":"2023-07-06T21:39:27.741202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:24.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740436+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:39:27.740436+0000","last_peered":"2023-07-06T21:39:27.740436+0000","last_clean":"2023-07-06T21:39:27.740436+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:39:27.740436+0000","last_undegraded":"2023-07-06T21:39:27.740436+0000","last_fullsized":"2023-07-06T21:39:27.740436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:07.704609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439168+0000","last_change":"2023-07-06T21:39:25.962785+0000","last_active":"2023-07-06T21:39:27.439168+0000","last_peered":"2023-07-06T21:39:27.439168+0000","last_clean":"2023-07-06T21:39:27.439168+0000","last_became_active":"2023-07-06T21:39:25.962456+0000","last_became_peered":"2023-07-06T21:39:25.962456+0000","last_unstale":"2023-07-06T21:39:27.439168+0000","last_undegraded":"2023-07-06T21:39:27.439168+0000","last_fullsized":"2023-07-06T21:39:27.439168+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:50:54.935056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740463+0000","last_change":"2023-07-06T21:39:26.457017+0000","last_active":"2023-07-06T21:39:27.740463+0000","last_peered":"2023-07-06T21:39:27.740463+0000","last_clean":"2023-07-06T21:39:27.740463+0000","last_became_active":"2023-07-06T21:39:26.456767+0000","last_became_peered":"2023-07-06T21:39:26.456767+0000","last_unstale":"2023-07-06T21:39:27.740463+0000","last_undegraded":"2023-07-06T21:39:27.740463+0000","last_fullsized":"2023-07-06T21:39:27.740463+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:56.153818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438825+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:39:27.438825+0000","last_peered":"2023-07-06T21:39:27.438825+0000","last_clean":"2023-07-06T21:39:27.438825+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:39:27.438825+0000","last_undegraded":"2023-07-06T21:39:27.438825+0000","last_fullsized":"2023-07-06T21:39:27.438825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:11.309216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740664+0000","last_change":"2023-07-06T21:39:25.955813+0000","last_active":"2023-07-06T21:39:27.740664+0000","last_peered":"2023-07-06T21:39:27.740664+0000","last_clean":"2023-07-06T21:39:27.740664+0000","last_became_active":"2023-07-06T21:39:25.955570+0000","last_became_peered":"2023-07-06T21:39:25.955570+0000","last_unstale":"2023-07-06T21:39:27.740664+0000","last_undegraded":"2023-07-06T21:39:27.740664+0000","last_fullsized":"2023-07-06T21:39:27.740664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:13.852251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740595+0000","last_change":"2023-07-06T21:39:26.458071+0000","last_active":"2023-07-06T21:39:27.740595+0000","last_peered":"2023-07-06T21:39:27.740595+0000","last_clean":"2023-07-06T21:39:27.740595+0000","last_became_active":"2023-07-06T21:39:26.457809+0000","last_became_peered":"2023-07-06T21:39:26.457809+0000","last_unstale":"2023-07-06T21:39:27.740595+0000","last_undegraded":"2023-07-06T21:39:27.740595+0000","last_fullsized":"2023-07-06T21:39:27.740595+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:17.735656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739565+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:39:27.739565+0000","last_peered":"2023-07-06T21:39:27.739565+0000","last_clean":"2023-07-06T21:39:27.739565+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:39:27.739565+0000","last_undegraded":"2023-07-06T21:39:27.739565+0000","last_fullsized":"2023-07-06T21:39:27.739565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:44:12.312730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740253+0000","last_change":"2023-07-06T21:39:25.951522+0000","last_active":"2023-07-06T21:39:27.740253+0000","last_peered":"2023-07-06T21:39:27.740253+0000","last_clean":"2023-07-06T21:39:27.740253+0000","last_became_active":"2023-07-06T21:39:25.951221+0000","last_became_peered":"2023-07-06T21:39:25.951221+0000","last_unstale":"2023-07-06T21:39:27.740253+0000","last_undegraded":"2023-07-06T21:39:27.740253+0000","last_fullsized":"2023-07-06T21:39:27.740253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:35:35.274536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741026+0000","last_change":"2023-07-06T21:39:26.456883+0000","last_active":"2023-07-06T21:39:27.741026+0000","last_peered":"2023-07-06T21:39:27.741026+0000","last_clean":"2023-07-06T21:39:27.741026+0000","last_became_active":"2023-07-06T21:39:26.456380+0000","last_became_peered":"2023-07-06T21:39:26.456380+0000","last_unstale":"2023-07-06T21:39:27.741026+0000","last_undegraded":"2023-07-06T21:39:27.741026+0000","last_fullsized":"2023-07-06T21:39:27.741026+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:22:14.860196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438695+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:39:27.438695+0000","last_peered":"2023-07-06T21:39:27.438695+0000","last_clean":"2023-07-06T21:39:27.438695+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:39:27.438695+0000","last_undegraded":"2023-07-06T21:39:27.438695+0000","last_fullsized":"2023-07-06T21:39:27.438695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:49.727377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":175,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739675+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:39:27.739675+0000","last_peered":"2023-07-06T21:39:27.739675+0000","last_clean":"2023-07-06T21:39:27.739675+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:39:27.739675+0000","last_undegraded":"2023-07-06T21:39:27.739675+0000","last_fullsized":"2023-07-06T21:39:27.739675+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:11.340670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437268+0000","last_change":"2023-07-06T21:39:25.953072+0000","last_active":"2023-07-06T21:39:27.437268+0000","last_peered":"2023-07-06T21:39:27.437268+0000","last_clean":"2023-07-06T21:39:27.437268+0000","last_became_active":"2023-07-06T21:39:25.952723+0000","last_became_peered":"2023-07-06T21:39:25.952723+0000","last_unstale":"2023-07-06T21:39:27.437268+0000","last_undegraded":"2023-07-06T21:39:27.437268+0000","last_fullsized":"2023-07-06T21:39:27.437268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:53.825665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438345+0000","last_change":"2023-07-06T21:39:26.459006+0000","last_active":"2023-07-06T21:39:27.438345+0000","last_peered":"2023-07-06T21:39:27.438345+0000","last_clean":"2023-07-06T21:39:27.438345+0000","last_became_active":"2023-07-06T21:39:26.458537+0000","last_became_peered":"2023-07-06T21:39:26.458537+0000","last_unstale":"2023-07-06T21:39:27.438345+0000","last_undegraded":"2023-07-06T21:39:27.438345+0000","last_fullsized":"2023-07-06T21:39:27.438345+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:59.208717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739820+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:39:27.739820+0000","last_peered":"2023-07-06T21:39:27.739820+0000","last_clean":"2023-07-06T21:39:27.739820+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:39:27.739820+0000","last_undegraded":"2023-07-06T21:39:27.739820+0000","last_fullsized":"2023-07-06T21:39:27.739820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:59.602755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739590+0000","last_change":"2023-07-06T21:39:25.951755+0000","last_active":"2023-07-06T21:39:27.739590+0000","last_peered":"2023-07-06T21:39:27.739590+0000","last_clean":"2023-07-06T21:39:27.739590+0000","last_became_active":"2023-07-06T21:39:25.951412+0000","last_became_peered":"2023-07-06T21:39:25.951412+0000","last_unstale":"2023-07-06T21:39:27.739590+0000","last_undegraded":"2023-07-06T21:39:27.739590+0000","last_fullsized":"2023-07-06T21:39:27.739590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:27.792618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741071+0000","last_change":"2023-07-06T21:39:26.455874+0000","last_active":"2023-07-06T21:39:27.741071+0000","last_peered":"2023-07-06T21:39:27.741071+0000","last_clean":"2023-07-06T21:39:27.741071+0000","last_became_active":"2023-07-06T21:39:26.455628+0000","last_became_peered":"2023-07-06T21:39:26.455628+0000","last_unstale":"2023-07-06T21:39:27.741071+0000","last_undegraded":"2023-07-06T21:39:27.741071+0000","last_fullsized":"2023-07-06T21:39:27.741071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:41:17.735656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740286+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:39:27.740286+0000","last_peered":"2023-07-06T21:39:27.740286+0000","last_clean":"2023-07-06T21:39:27.740286+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:39:27.740286+0000","last_undegraded":"2023-07-06T21:39:27.740286+0000","last_fullsized":"2023-07-06T21:39:27.740286+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:20.500360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740715+0000","last_change":"2023-07-06T21:39:25.959332+0000","last_active":"2023-07-06T21:39:27.740715+0000","last_peered":"2023-07-06T21:39:27.740715+0000","last_clean":"2023-07-06T21:39:27.740715+0000","last_became_active":"2023-07-06T21:39:25.958983+0000","last_became_peered":"2023-07-06T21:39:25.958983+0000","last_unstale":"2023-07-06T21:39:27.740715+0000","last_undegraded":"2023-07-06T21:39:27.740715+0000","last_fullsized":"2023-07-06T21:39:27.740715+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:14.791877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740002+0000","last_change":"2023-07-06T21:39:26.455969+0000","last_active":"2023-07-06T21:39:27.740002+0000","last_peered":"2023-07-06T21:39:27.740002+0000","last_clean":"2023-07-06T21:39:27.740002+0000","last_became_active":"2023-07-06T21:39:26.455716+0000","last_became_peered":"2023-07-06T21:39:26.455716+0000","last_unstale":"2023-07-06T21:39:27.740002+0000","last_undegraded":"2023-07-06T21:39:27.740002+0000","last_fullsized":"2023-07-06T21:39:27.740002+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437645+0000","last_change":"2023-07-06T21:39:25.962443+0000","last_active":"2023-07-06T21:39:27.437645+0000","last_peered":"2023-07-06T21:39:27.437645+0000","last_clean":"2023-07-06T21:39:27.437645+0000","last_became_active":"2023-07-06T21:39:25.962123+0000","last_became_peered":"2023-07-06T21:39:25.962123+0000","last_unstale":"2023-07-06T21:39:27.437645+0000","last_undegraded":"2023-07-06T21:39:27.437645+0000","last_fullsized":"2023-07-06T21:39:27.437645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:12.587813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740998+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:39:27.740998+0000","last_peered":"2023-07-06T21:39:27.740998+0000","last_clean":"2023-07-06T21:39:27.740998+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:39:27.740998+0000","last_undegraded":"2023-07-06T21:39:27.740998+0000","last_fullsized":"2023-07-06T21:39:27.740998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:28:59.602755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.436960+0000","last_change":"2023-07-06T21:39:26.463271+0000","last_active":"2023-07-06T21:39:27.436960+0000","last_peered":"2023-07-06T21:39:27.436960+0000","last_clean":"2023-07-06T21:39:27.436960+0000","last_became_active":"2023-07-06T21:39:26.463082+0000","last_became_peered":"2023-07-06T21:39:26.463082+0000","last_unstale":"2023-07-06T21:39:27.436960+0000","last_undegraded":"2023-07-06T21:39:27.436960+0000","last_fullsized":"2023-07-06T21:39:27.436960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:38:07.305855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438426+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:39:27.438426+0000","last_peered":"2023-07-06T21:39:27.438426+0000","last_clean":"2023-07-06T21:39:27.438426+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:39:27.438426+0000","last_undegraded":"2023-07-06T21:39:27.438426+0000","last_fullsized":"2023-07-06T21:39:27.438426+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:05:58.605808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.741247+0000","last_change":"2023-07-06T21:39:25.946579+0000","last_active":"2023-07-06T21:39:27.741247+0000","last_peered":"2023-07-06T21:39:27.741247+0000","last_clean":"2023-07-06T21:39:27.741247+0000","last_became_active":"2023-07-06T21:39:25.945265+0000","last_became_peered":"2023-07-06T21:39:25.945265+0000","last_unstale":"2023-07-06T21:39:27.741247+0000","last_undegraded":"2023-07-06T21:39:27.741247+0000","last_fullsized":"2023-07-06T21:39:27.741247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:14.791877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740077+0000","last_change":"2023-07-06T21:39:26.458730+0000","last_active":"2023-07-06T21:39:27.740077+0000","last_peered":"2023-07-06T21:39:27.740077+0000","last_clean":"2023-07-06T21:39:27.740077+0000","last_became_active":"2023-07-06T21:39:26.458438+0000","last_became_peered":"2023-07-06T21:39:26.458438+0000","last_unstale":"2023-07-06T21:39:27.740077+0000","last_undegraded":"2023-07-06T21:39:27.740077+0000","last_fullsized":"2023-07-06T21:39:27.740077+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740082+0000","last_change":"2023-07-06T21:39:25.948372+0000","last_active":"2023-07-06T21:39:27.740082+0000","last_peered":"2023-07-06T21:39:27.740082+0000","last_clean":"2023-07-06T21:39:27.740082+0000","last_became_active":"2023-07-06T21:39:25.948140+0000","last_became_peered":"2023-07-06T21:39:25.948140+0000","last_unstale":"2023-07-06T21:39:27.740082+0000","last_undegraded":"2023-07-06T21:39:27.740082+0000","last_fullsized":"2023-07-06T21:39:27.740082+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:54:10.672168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739884+0000","last_change":"2023-07-06T21:39:26.454713+0000","last_active":"2023-07-06T21:39:27.739884+0000","last_peered":"2023-07-06T21:39:27.739884+0000","last_clean":"2023-07-06T21:39:27.739884+0000","last_became_active":"2023-07-06T21:39:26.454273+0000","last_became_peered":"2023-07-06T21:39:26.454273+0000","last_unstale":"2023-07-06T21:39:27.739884+0000","last_undegraded":"2023-07-06T21:39:27.739884+0000","last_fullsized":"2023-07-06T21:39:27.739884+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:44.190028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438831+0000","last_change":"2023-07-06T21:39:25.951864+0000","last_active":"2023-07-06T21:39:27.438831+0000","last_peered":"2023-07-06T21:39:27.438831+0000","last_clean":"2023-07-06T21:39:27.438831+0000","last_became_active":"2023-07-06T21:39:25.951527+0000","last_became_peered":"2023-07-06T21:39:25.951527+0000","last_unstale":"2023-07-06T21:39:27.438831+0000","last_undegraded":"2023-07-06T21:39:27.438831+0000","last_fullsized":"2023-07-06T21:39:27.438831+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:39:23.569276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740558+0000","last_change":"2023-07-06T21:39:26.450910+0000","last_active":"2023-07-06T21:39:27.740558+0000","last_peered":"2023-07-06T21:39:27.740558+0000","last_clean":"2023-07-06T21:39:27.740558+0000","last_became_active":"2023-07-06T21:39:26.450613+0000","last_became_peered":"2023-07-06T21:39:26.450613+0000","last_unstale":"2023-07-06T21:39:27.740558+0000","last_undegraded":"2023-07-06T21:39:27.740558+0000","last_fullsized":"2023-07-06T21:39:27.740558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:30:13.595758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.439232+0000","last_change":"2023-07-06T21:39:25.945875+0000","last_active":"2023-07-06T21:39:27.439232+0000","last_peered":"2023-07-06T21:39:27.439232+0000","last_clean":"2023-07-06T21:39:27.439232+0000","last_became_active":"2023-07-06T21:39:25.945533+0000","last_became_peered":"2023-07-06T21:39:25.945533+0000","last_unstale":"2023-07-06T21:39:27.439232+0000","last_undegraded":"2023-07-06T21:39:27.439232+0000","last_fullsized":"2023-07-06T21:39:27.439232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:25.918856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740487+0000","last_change":"2023-07-06T21:39:26.447102+0000","last_active":"2023-07-06T21:39:27.740487+0000","last_peered":"2023-07-06T21:39:27.740487+0000","last_clean":"2023-07-06T21:39:27.740487+0000","last_became_active":"2023-07-06T21:39:26.446750+0000","last_became_peered":"2023-07-06T21:39:26.446750+0000","last_unstale":"2023-07-06T21:39:27.740487+0000","last_undegraded":"2023-07-06T21:39:27.740487+0000","last_fullsized":"2023-07-06T21:39:27.740487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:11:15.799821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437289+0000","last_change":"2023-07-06T21:39:25.952004+0000","last_active":"2023-07-06T21:39:27.437289+0000","last_peered":"2023-07-06T21:39:27.437289+0000","last_clean":"2023-07-06T21:39:27.437289+0000","last_became_active":"2023-07-06T21:39:25.951708+0000","last_became_peered":"2023-07-06T21:39:25.951708+0000","last_unstale":"2023-07-06T21:39:27.437289+0000","last_undegraded":"2023-07-06T21:39:27.437289+0000","last_fullsized":"2023-07-06T21:39:27.437289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:13.123105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437012+0000","last_change":"2023-07-06T21:39:26.450232+0000","last_active":"2023-07-06T21:39:27.437012+0000","last_peered":"2023-07-06T21:39:27.437012+0000","last_clean":"2023-07-06T21:39:27.437012+0000","last_became_active":"2023-07-06T21:39:26.449627+0000","last_became_peered":"2023-07-06T21:39:26.449627+0000","last_unstale":"2023-07-06T21:39:27.437012+0000","last_undegraded":"2023-07-06T21:39:27.437012+0000","last_fullsized":"2023-07-06T21:39:27.437012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740957+0000","last_change":"2023-07-06T21:39:25.945282+0000","last_active":"2023-07-06T21:39:27.740957+0000","last_peered":"2023-07-06T21:39:27.740957+0000","last_clean":"2023-07-06T21:39:27.740957+0000","last_became_active":"2023-07-06T21:39:25.944898+0000","last_became_peered":"2023-07-06T21:39:25.944898+0000","last_unstale":"2023-07-06T21:39:27.740957+0000","last_undegraded":"2023-07-06T21:39:27.740957+0000","last_fullsized":"2023-07-06T21:39:27.740957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:24.915940+0000","last_clean_scrub_stamp":"2023-07-06T21:39:24.915940+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:55.145873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.739887+0000","last_change":"2023-07-06T21:39:26.470499+0000","last_active":"2023-07-06T21:39:27.739887+0000","last_peered":"2023-07-06T21:39:27.739887+0000","last_clean":"2023-07-06T21:39:27.739887+0000","last_became_active":"2023-07-06T21:39:26.470270+0000","last_became_peered":"2023-07-06T21:39:26.470270+0000","last_unstale":"2023-07-06T21:39:27.739887+0000","last_undegraded":"2023-07-06T21:39:27.739887+0000","last_fullsized":"2023-07-06T21:39:27.739887+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:14.990196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.438288+0000","last_change":"2023-07-06T21:39:26.469954+0000","last_active":"2023-07-06T21:39:27.438288+0000","last_peered":"2023-07-06T21:39:27.438288+0000","last_clean":"2023-07-06T21:39:27.438288+0000","last_became_active":"2023-07-06T21:39:26.469618+0000","last_became_peered":"2023-07-06T21:39:26.469618+0000","last_unstale":"2023-07-06T21:39:27.438288+0000","last_undegraded":"2023-07-06T21:39:27.438288+0000","last_fullsized":"2023-07-06T21:39:27.438288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:38:33.954720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437005+0000","last_change":"2023-07-06T21:39:26.450370+0000","last_active":"2023-07-06T21:39:27.437005+0000","last_peered":"2023-07-06T21:39:27.437005+0000","last_clean":"2023-07-06T21:39:27.437005+0000","last_became_active":"2023-07-06T21:39:26.449980+0000","last_became_peered":"2023-07-06T21:39:26.449980+0000","last_unstale":"2023-07-06T21:39:27.437005+0000","last_undegraded":"2023-07-06T21:39:27.437005+0000","last_fullsized":"2023-07-06T21:39:27.437005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:49:57.797206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740231+0000","last_change":"2023-07-06T21:39:26.469630+0000","last_active":"2023-07-06T21:39:27.740231+0000","last_peered":"2023-07-06T21:39:27.740231+0000","last_clean":"2023-07-06T21:39:27.740231+0000","last_became_active":"2023-07-06T21:39:26.469358+0000","last_became_peered":"2023-07-06T21:39:26.469358+0000","last_unstale":"2023-07-06T21:39:27.740231+0000","last_undegraded":"2023-07-06T21:39:27.740231+0000","last_fullsized":"2023-07-06T21:39:27.740231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:46:14.131050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.437564+0000","last_change":"2023-07-06T21:39:26.466945+0000","last_active":"2023-07-06T21:39:27.437564+0000","last_peered":"2023-07-06T21:39:27.437564+0000","last_clean":"2023-07-06T21:39:27.437564+0000","last_became_active":"2023-07-06T21:39:26.466613+0000","last_became_peered":"2023-07-06T21:39:26.466613+0000","last_unstale":"2023-07-06T21:39:27.437564+0000","last_undegraded":"2023-07-06T21:39:27.437564+0000","last_fullsized":"2023-07-06T21:39:27.437564+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:53:24.919738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-07-06T21:39:27.740297+0000","last_change":"2023-07-06T21:39:26.466793+0000","last_active":"2023-07-06T21:39:27.740297+0000","last_peered":"2023-07-06T21:39:27.740297+0000","last_clean":"2023-07-06T21:39:27.740297+0000","last_became_active":"2023-07-06T21:39:26.466585+0000","last_became_peered":"2023-07-06T21:39:26.466585+0000","last_unstale":"2023-07-06T21:39:27.740297+0000","last_undegraded":"2023-07-06T21:39:27.740297+0000","last_fullsized":"2023-07-06T21:39:27.740297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:25.923884+0000","last_clean_scrub_stamp":"2023-07-06T21:39:25.923884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:55:53.552232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":864256,"data_stored":681747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":816,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":835584,"data_stored":679530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7869999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3319999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3440000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.234}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.544}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.395}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2629999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":241664,"data_stored":89599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81499999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":245760,"data_stored":89689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":262144,"data_stored":91219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":249856,"data_stored":90251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:39:37.473 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T21:39:37.473 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-06T21:39:37.473 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-07-06T21:39:37.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-07-06T21:39:37.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.696+0000 7ff8b6bff700 1 Processor -- start 2023-07-06T21:39:37.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.696+0000 7ff8b6bff700 1 -- start start 2023-07-06T21:39:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.697+0000 7ff8b6bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff8b00d0a10 0x7ff8b00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.697+0000 7ff8b6bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8b00cd220 0x7ff8b00d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.697+0000 7ff8b6bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b00d23d0 con 0x7ff8b00d0de0 2023-07-06T21:39:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.697+0000 7ff8b6bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b015c8c0 con 0x7ff8b00cd220 2023-07-06T21:39:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.697+0000 7ff8b6bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b00d2620 con 0x7ff8b00d0a10 2023-07-06T21:39:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8b00cd220 0x7ff8b00d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff8b00cd220 0x7ff8b00d33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51148/0 (socket says 172.21.15.84:51148) 2023-07-06T21:39:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 -- 172.21.15.84:0/4002541960 learned_addr learned my addr 172.21.15.84:0/4002541960 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8b499b700 1 --2- 172.21.15.84:0/4002541960 >> v2:172.21.15.107:3300/0 conn(0x7ff8b00d0a10 0x7ff8b00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 -- 172.21.15.84:0/4002541960 >> v2:172.21.15.107:3300/0 conn(0x7ff8b00d0a10 msgr2=0x7ff8b00cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 --2- 172.21.15.84:0/4002541960 >> v2:172.21.15.107:3300/0 conn(0x7ff8b00d0a10 0x7ff8b00cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 -- 172.21.15.84:0/4002541960 >> v1:172.21.15.107:6789/0 conn(0x7ff8b00d0de0 legacy=0x7ff8b00b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 -- 172.21.15.84:0/4002541960 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00d2900 con 0x7ff8b00cd220 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8b499b700 1 --2- 172.21.15.84:0/4002541960 >> v2:172.21.15.107:3300/0 conn(0x7ff8b00d0a10 0x7ff8b00cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8affff700 1 --2- 172.21.15.84:0/4002541960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00cd220 0x7ff8b00d33e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8b008be50 tx=0x7ff89c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc8625ddc44f589 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8af7fe700 1 -- 172.21.15.84:0/4002541960 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c004230 con 0x7ff8b00cd220 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.698+0000 7ff8af7fe700 1 -- 172.21.15.84:0/4002541960 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c004390 con 0x7ff8b00cd220 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8af7fe700 1 -- 172.21.15.84:0/4002541960 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0036a0 con 0x7ff8b00cd220 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 -- 172.21.15.84:0/4002541960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00cd220 msgr2=0x7ff8b00d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/4002541960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00cd220 0x7ff8b00d33e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff8b008be50 tx=0x7ff89c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 -- 172.21.15.84:0/4002541960 shutdown_connections 2023-07-06T21:39:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/4002541960 >> v2:172.21.15.107:3300/0 conn(0x7ff8b00d0a10 0x7ff8b00cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/4002541960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00cd220 0x7ff8b00d33e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 -- 172.21.15.84:0/4002541960 >> 172.21.15.84:0/4002541960 conn(0x7ff8b00715c0 msgr2=0x7ff8b0071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 -- 172.21.15.84:0/4002541960 shutdown_connections 2023-07-06T21:39:37.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.699+0000 7ff8b6bff700 1 -- 172.21.15.84:0/4002541960 wait complete. 2023-07-06T21:39:37.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.701+0000 7ff8b6bff700 1 Processor -- start 2023-07-06T21:39:37.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.701+0000 7ff8b6bff700 1 -- start start 2023-07-06T21:39:37.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b6bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8b00cd220 0x7ff8b015e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b6bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 0x7ff8b00c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b6bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b00d0de0 0x7ff8b015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b6bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00c9590 con 0x7ff8b00d0a10 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b6bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b00c92b0 con 0x7ff8b00d0de0 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b6bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b015cb10 con 0x7ff8b00cd220 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8affff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 0x7ff8b00c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8affff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 0x7ff8b00c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51152/0 (socket says 172.21.15.84:51152) 2023-07-06T21:39:37.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8affff700 1 -- 172.21.15.84:0/515604856 learned_addr learned my addr 172.21.15.84:0/515604856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:37.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b499b700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8b00cd220 0x7ff8b015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.702+0000 7ff8b519c700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b00d0de0 0x7ff8b015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8affff700 1 -- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8b00cd220 msgr2=0x7ff8b015e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8affff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8b00cd220 0x7ff8b015e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8affff700 1 -- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b00d0de0 msgr2=0x7ff8b015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8affff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b00d0de0 0x7ff8b015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8affff700 1 -- 172.21.15.84:0/515604856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c009550 con 0x7ff8b00d0a10 2023-07-06T21:39:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8affff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 0x7ff8b00c66c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009130 tx=0x7ff89c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c003b90 con 0x7ff8b00d0a10 2023-07-06T21:39:37.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c01f070 con 0x7ff8b00d0a10 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b015cdf0 con 0x7ff8b00d0a10 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c011630 con 0x7ff8b00d0a10 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.703+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b015e870 con 0x7ff8b00d0a10 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.704+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b015d310 con 0x7ff8b00d0a10 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.705+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff89c021030 con 0x7ff8b00d0a10 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.707+0000 7ff8ad7fa700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff89406ace0 0x7ff89406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:37.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.707+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff89c08a940 con 0x7ff8b00d0a10 2023-07-06T21:39:37.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.707+0000 7ff8b499b700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff89406ace0 0x7ff89406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:37.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.707+0000 7ff8b499b700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff89406ace0 0x7ff89406d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0159b70 tx=0x7ff8a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:37.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.710+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff8b015d310 con 0x7ff8b00d0a10 2023-07-06T21:39:37.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.948+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff8b015d310 con 0x7ff8b00d0a10 2023-07-06T21:39:37.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.949+0000 7ff8ad7fa700 1 -- 172.21.15.84:0/515604856 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff8b015d310 con 0x7ff8b00d0a10 2023-07-06T21:39:37.952 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:37.952 INFO:teuthology.orchestra.run.smithi084.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-06T21:39:37.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff89406ace0 msgr2=0x7ff89406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff89406ace0 0x7ff89406d190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0159b70 tx=0x7ff8a4008040 comp rx=0 tx=0).stop 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 msgr2=0x7ff8b00c66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 0x7ff8b00c66c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009130 tx=0x7ff89c004750 comp rx=0 tx=0).stop 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 shutdown_connections 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff89406ace0 0x7ff89406d190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8b00d0de0 0x7ff8b015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff8b00cd220 0x7ff8b015e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 --2- 172.21.15.84:0/515604856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff8b00d0a10 0x7ff8b00c66c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 >> 172.21.15.84:0/515604856 conn(0x7ff8b00715c0 msgr2=0x7ff8b00b5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 shutdown_connections 2023-07-06T21:39:37.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:37.951+0000 7ff8b6bff700 1 -- 172.21.15.84:0/515604856 wait complete. 2023-07-06T21:39:37.972 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-07-06T21:39:37.972 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:38.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.163+0000 7fc8053ff700 1 Processor -- start 2023-07-06T21:39:38.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.163+0000 7fc8053ff700 1 -- start start 2023-07-06T21:39:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.164+0000 7fc8053ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000b83c0 0x7fc8000cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc8053ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8000cfc60 0x7fc8000d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc8053ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000d0a80 con 0x7fc8000d3710 2023-07-06T21:39:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc8053ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8000b8790 con 0x7fc8000cfc60 2023-07-06T21:39:38.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc8053ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000b4e70 con 0x7fc8000b83c0 2023-07-06T21:39:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7f7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc8000d3710 0x7fc8000cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47872/0 (socket says 172.21.15.84:47872) 2023-07-06T21:39:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7f7fff700 1 -- 172.21.15.84:0/1559584846 learned_addr learned my addr 172.21.15.84:0/1559584846 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7ff7fe700 1 --2- 172.21.15.84:0/1559584846 >> v2:172.21.15.84:3300/0 conn(0x7fc8000cfc60 0x7fc8000d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7feffd700 1 --2- 172.21.15.84:0/1559584846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000b83c0 0x7fc8000cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7fe7fc700 1 -- 172.21.15.84:0/1559584846 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364904680 0 0) 0x7fc8000d0a80 con 0x7fc8000d3710 2023-07-06T21:39:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7fe7fc700 1 -- 172.21.15.84:0/1559584846 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7ec0035b0 con 0x7fc8000d3710 2023-07-06T21:39:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7ff7fe700 1 -- 172.21.15.84:0/1559584846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000b83c0 msgr2=0x7fc8000cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7ff7fe700 1 --2- 172.21.15.84:0/1559584846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000b83c0 0x7fc8000cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7ff7fe700 1 -- 172.21.15.84:0/1559584846 >> v1:172.21.15.84:6789/0 conn(0x7fc8000d3710 legacy=0x7fc8000cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.165+0000 7fc7ff7fe700 1 -- 172.21.15.84:0/1559584846 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000b5150 con 0x7fc8000cfc60 2023-07-06T21:39:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc7ff7fe700 1 --2- 172.21.15.84:0/1559584846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000cfc60 0x7fc8000d0f10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009d90 tx=0x7fc7f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e599a9087fedf39b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc7fe7fc700 1 -- 172.21.15.84:0/1559584846 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800e070 con 0x7fc8000cfc60 2023-07-06T21:39:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc7fe7fc700 1 -- 172.21.15.84:0/1559584846 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f800bb20 con 0x7fc8000cfc60 2023-07-06T21:39:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc7fe7fc700 1 -- 172.21.15.84:0/1559584846 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8012430 con 0x7fc8000cfc60 2023-07-06T21:39:38.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc8053ff700 1 -- 172.21.15.84:0/1559584846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000cfc60 msgr2=0x7fc8000d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc8053ff700 1 --2- 172.21.15.84:0/1559584846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000cfc60 0x7fc8000d0f10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009d90 tx=0x7fc7f800b330 comp rx=0 tx=0).stop 2023-07-06T21:39:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc8053ff700 1 -- 172.21.15.84:0/1559584846 shutdown_connections 2023-07-06T21:39:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc8053ff700 1 --2- 172.21.15.84:0/1559584846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000cfc60 0x7fc8000d0f10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc8053ff700 1 --2- 172.21.15.84:0/1559584846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000b83c0 0x7fc8000cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.166+0000 7fc8053ff700 1 -- 172.21.15.84:0/1559584846 >> 172.21.15.84:0/1559584846 conn(0x7fc80006b350 msgr2=0x7fc80006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:38.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.167+0000 7fc8053ff700 1 -- 172.21.15.84:0/1559584846 shutdown_connections 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.167+0000 7fc8053ff700 1 -- 172.21.15.84:0/1559584846 wait complete. 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.169+0000 7fc8053ff700 1 Processor -- start 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.169+0000 7fc8053ff700 1 -- start start 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc8053ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 0x7fc80015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc8053ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8000cfc60 0x7fc800156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc8053ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000d3710 0x7fc800153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc8053ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001f5df0 con 0x7fc8000b83c0 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc8053ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001f4f00 con 0x7fc8000cfc60 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc8053ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001f51e0 con 0x7fc8000d3710 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.170+0000 7fc7feffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 0x7fc80015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 0x7fc80015aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51166/0 (socket says 172.21.15.84:51166) 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 -- 172.21.15.84:0/3261360547 learned_addr learned my addr 172.21.15.84:0/3261360547 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7ff7fe700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000d3710 0x7fc800153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7f7fff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8000cfc60 0x7fc800156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 -- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000d3710 msgr2=0x7fc800153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000d3710 0x7fc800153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 -- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8000cfc60 msgr2=0x7fc800156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8000cfc60 0x7fc800156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 -- 172.21.15.84:0/3261360547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8009a70 con 0x7fc8000b83c0 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7f7fff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8000cfc60 0x7fc800156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7feffd700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 0x7fc80015aa40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000e870 tx=0x7fc7f000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0009590 con 0x7fc8000b83c0 2023-07-06T21:39:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f0013680 con 0x7fc8000b83c0 2023-07-06T21:39:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f00139f0 con 0x7fc8000b83c0 2023-07-06T21:39:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.171+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8001f54c0 con 0x7fc8000b83c0 2023-07-06T21:39:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.172+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8001f5a80 con 0x7fc8000b83c0 2023-07-06T21:39:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.172+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80015c490 con 0x7fc8000b83c0 2023-07-06T21:39:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.173+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc7f0013b50 con 0x7fc8000b83c0 2023-07-06T21:39:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.174+0000 7fc7f77fe700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc7e006ace0 0x7fc7e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.174+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc7f0089430 con 0x7fc8000b83c0 2023-07-06T21:39:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.174+0000 7fc7f7fff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc7e006ace0 0x7fc7e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.175+0000 7fc7f7fff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc7e006ace0 0x7fc7e006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc80015b450 tx=0x7fc7e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.178+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc7f0054440 con 0x7fc8000b83c0 2023-07-06T21:39:38.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.372+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8000d0710 con 0x7fc8000b83c0 2023-07-06T21:39:38.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.373+0000 7fc7f77fe700 1 -- 172.21.15.84:0/3261360547 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7fc7f0057a60 con 0x7fc8000b83c0 2023-07-06T21:39:38.377 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:38.378 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:38.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.375+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc7e006ace0 msgr2=0x7fc7e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.375+0000 7fc8053ff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc7e006ace0 0x7fc7e006d190 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc80015b450 tx=0x7fc7e8009040 comp rx=0 tx=0).stop 2023-07-06T21:39:38.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.375+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 msgr2=0x7fc80015aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.375+0000 7fc8053ff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 0x7fc80015aa40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000e870 tx=0x7fc7f000eb80 comp rx=0 tx=0).stop 2023-07-06T21:39:38.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 shutdown_connections 2023-07-06T21:39:38.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc7e006ace0 0x7fc7e006d190 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8000d3710 0x7fc800153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8000b83c0 0x7fc80015aa40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 --2- 172.21.15.84:0/3261360547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8000cfc60 0x7fc800156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 >> 172.21.15.84:0/3261360547 conn(0x7fc80006b350 msgr2=0x7fc8000d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:38.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 shutdown_connections 2023-07-06T21:39:38.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.376+0000 7fc8053ff700 1 -- 172.21.15.84:0/3261360547 wait complete. 2023-07-06T21:39:38.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:39:38.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:39:38.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.593+0000 7f7f0d1ff700 1 Processor -- start 2023-07-06T21:39:38.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.593+0000 7f7f0d1ff700 1 -- start start 2023-07-06T21:39:38.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.594+0000 7f7f0d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f080cfd10 0x7f7f080d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f0d1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080d1830 con 0x7f7f080b1a80 2023-07-06T21:39:38.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f0d1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f080cf330 con 0x7f7f080b81c0 2023-07-06T21:39:38.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f0d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080d1270 con 0x7f7f080cfd10 2023-07-06T21:39:38.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f0759e700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7f080b1a80 0x7f7f080d28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47874/0 (socket says 172.21.15.84:47874) 2023-07-06T21:39:38.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f0759e700 1 -- 172.21.15.84:0/1900584957 learned_addr learned my addr 172.21.15.84:0/1900584957 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:38.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f0659c700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f080cfd10 0x7f7f080d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597436531 0 0) 0x7f7f080d1830 con 0x7f7f080b1a80 2023-07-06T21:39:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ef00035b0 con 0x7f7f080b1a80 2023-07-06T21:39:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871103818 0 0) 0x7f7f080cf330 con 0x7f7f080b81c0 2023-07-06T21:39:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f080d1830 con 0x7f7f080b81c0 2023-07-06T21:39:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3532554562 0 0) 0x7f7ef00035b0 con 0x7f7f080b1a80 2023-07-06T21:39:38.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f080cfd10 msgr2=0x7f7f080d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f080cfd10 0x7f7f080d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 >> v1:172.21.15.107:6789/0 conn(0x7f7f080b81c0 legacy=0x7f7f080cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080d1550 con 0x7f7f080b1a80 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.595+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f7f000019f0 con 0x7f7f080b1a80 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 >> v1:172.21.15.84:6789/0 conn(0x7f7f080b1a80 legacy=0x7f7f080d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0003bc0 0x7f7ef0003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0004850 0x7f7ef0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef0005200 0x7f7ef000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080d1270 con 0x7f7ef0004850 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef000bfd0 con 0x7f7ef0005200 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef000c190 con 0x7f7ef0003bc0 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f080cfd10 0x7f7f080d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0004850 0x7f7ef0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f06d9d700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0003bc0 0x7f7ef0003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0759e700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef0005200 0x7f7ef000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 -- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0003bc0 msgr2=0x7f7ef0003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0003bc0 0x7f7ef0003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 -- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef0005200 msgr2=0x7f7ef000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef0005200 0x7f7ef000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0659c700 1 -- 172.21.15.84:0/1900584957 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef00035b0 con 0x7f7ef0004850 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f06d9d700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0003bc0 0x7f7ef0003fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.596+0000 7f7f0759e700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef0005200 0x7f7ef000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:38.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0659c700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0004850 0x7f7ef0004cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f0815fe80 tx=0x7f7ef0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f352d2ddb9fa36 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0096d0 con 0x7f7ef0004850 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc004030 con 0x7f7ef0004850 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f05d9b700 1 -- 172.21.15.84:0/1900584957 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0043e0 con 0x7f7ef0004850 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/1900584957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0004850 msgr2=0x7f7ef0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/1900584957 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ef0004850 0x7f7ef0004cc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7f0815fe80 tx=0x7f7ef0005880 comp rx=0 tx=0).stop 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0759e700 1 -- 172.21.15.84:0/1900584957 reap_dead start 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/1900584957 shutdown_connections 2023-07-06T21:39:38.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/1900584957 >> 172.21.15.84:0/1900584957 conn(0x7f7f0806b2b0 msgr2=0x7f7f0806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/1900584957 shutdown_connections 2023-07-06T21:39:38.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.597+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/1900584957 wait complete. 2023-07-06T21:39:38.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.600+0000 7f7f0d1ff700 1 Processor -- start 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.600+0000 7f7f0d1ff700 1 -- start start 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0004850 0x7f7f0815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0d1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 0x7f7f08151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0d1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f081e7d30 0x7f7f081e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0d1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f081e8710 con 0x7f7f0815b400 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0d1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f080cc8b0 con 0x7f7f081e7d30 2023-07-06T21:39:38.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0815f100 con 0x7f7ef0004850 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.601+0000 7f7f0659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 0x7f7f08151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f06d9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0004850 0x7f7f0815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 0x7f7f08151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51182/0 (socket says 172.21.15.84:51182) 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 -- 172.21.15.84:0/879855661 learned_addr learned my addr 172.21.15.84:0/879855661 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0759e700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f081e7d30 0x7f7f081e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 -- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0004850 msgr2=0x7f7f0815e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0004850 0x7f7f0815e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 -- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f081e7d30 msgr2=0x7f7f081e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f081e7d30 0x7f7f081e81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 -- 172.21.15.84:0/879855661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080ca4b0 con 0x7f7f0815b400 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0659c700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 0x7f7f08151c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000bfa0 tx=0x7f7efc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00eaa0 con 0x7f7f0815b400 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00ec00 con 0x7f7f0815b400 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0096d0 con 0x7f7f0815b400 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.602+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f08152630 con 0x7f7f0815b400 2023-07-06T21:39:38.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.603+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0815ee20 con 0x7f7f0815b400 2023-07-06T21:39:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.604+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f080ca640 con 0x7f7f0815b400 2023-07-06T21:39:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.605+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7efc013470 con 0x7f7f0815b400 2023-07-06T21:39:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.605+0000 7f7eef7fe700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7ef406ace0 0x7f7ef406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.606+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7efc08aab0 con 0x7f7f0815b400 2023-07-06T21:39:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.606+0000 7f7f06d9d700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7ef406ace0 0x7f7ef406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.606+0000 7f7f06d9d700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7ef406ace0 0x7f7ef406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000f60 tx=0x7f7ef8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:38.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.609+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7efc055bf0 con 0x7f7f0815b400 2023-07-06T21:39:38.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.804+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f0804e1d0 con 0x7f7f0815b400 2023-07-06T21:39:38.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.805+0000 7f7eef7fe700 1 -- 172.21.15.84:0/879855661 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7efc059210 con 0x7f7f0815b400 2023-07-06T21:39:38.809 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:39:38.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7ef406ace0 msgr2=0x7f7ef406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7ef406ace0 0x7f7ef406d190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8000f60 tx=0x7f7ef8009040 comp rx=0 tx=0).stop 2023-07-06T21:39:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 msgr2=0x7f7f08151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 0x7f7f08151c30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000bfa0 tx=0x7f7efc0047c0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 shutdown_connections 2023-07-06T21:39:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7ef406ace0 0x7f7ef406d190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f081e7d30 0x7f7f081e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ef0004850 0x7f7f0815e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 --2- 172.21.15.84:0/879855661 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f0815b400 0x7f7f08151c30 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:38.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 >> 172.21.15.84:0/879855661 conn(0x7f7f0806b2b0 msgr2=0x7f7f080b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:38.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 shutdown_connections 2023-07-06T21:39:38.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:38.807+0000 7f7f0d1ff700 1 -- 172.21.15.84:0/879855661 wait complete. 2023-07-06T21:39:38.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:39:39.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.024+0000 7fd1f2fff700 1 Processor -- start 2023-07-06T21:39:39.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.024+0000 7fd1f2fff700 1 -- start start 2023-07-06T21:39:39.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.025+0000 7fd1f2fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1ec0b81c0 0x7fd1ec0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1f2fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1ec0b1a80 0x7fd1ec0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1f2fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0d1830 con 0x7fd1ec0ceda0 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1f2fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0cf330 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1f2fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0d1270 con 0x7fd1ec0b81c0 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1ec0b1a80 0x7fd1ec0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd1ec0b1a80 0x7fd1ec0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51186/0 (socket says 172.21.15.84:51186) 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 -- 172.21.15.84:0/3620929870 learned_addr learned my addr 172.21.15.84:0/3620929870 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1f0d9b700 1 --2- 172.21.15.84:0/3620929870 >> v2:172.21.15.107:3300/0 conn(0x7fd1ec0b81c0 0x7fd1ec0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 -- 172.21.15.84:0/3620929870 >> v2:172.21.15.107:3300/0 conn(0x7fd1ec0b81c0 msgr2=0x7fd1ec0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:39.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 --2- 172.21.15.84:0/3620929870 >> v2:172.21.15.107:3300/0 conn(0x7fd1ec0b81c0 0x7fd1ec0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 -- 172.21.15.84:0/3620929870 >> v1:172.21.15.107:6789/0 conn(0x7fd1ec0ceda0 legacy=0x7fd1ec15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 -- 172.21.15.84:0/3620929870 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0d1550 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.026+0000 7fd1ebfff700 1 --2- 172.21.15.84:0/3620929870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec0cfb40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0b2620 tx=0x7fd1d8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01c4847a12ce120 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1eb7fe700 1 -- 172.21.15.84:0/3620929870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800b870 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1eb7fe700 1 -- 172.21.15.84:0/3620929870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d800b9d0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1eb7fe700 1 -- 172.21.15.84:0/3620929870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d800bd60 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 -- 172.21.15.84:0/3620929870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 msgr2=0x7fd1ec0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/3620929870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec0cfb40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0b2620 tx=0x7fd1d8009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 -- 172.21.15.84:0/3620929870 shutdown_connections 2023-07-06T21:39:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/3620929870 >> v2:172.21.15.107:3300/0 conn(0x7fd1ec0b81c0 0x7fd1ec0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/3620929870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec0cfb40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 -- 172.21.15.84:0/3620929870 >> 172.21.15.84:0/3620929870 conn(0x7fd1ec06b2b0 msgr2=0x7fd1ec06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:39.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 -- 172.21.15.84:0/3620929870 shutdown_connections 2023-07-06T21:39:39.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.027+0000 7fd1f2fff700 1 -- 172.21.15.84:0/3620929870 wait complete. 2023-07-06T21:39:39.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.029+0000 7fd1f2fff700 1 Processor -- start 2023-07-06T21:39:39.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.029+0000 7fd1f2fff700 1 -- start start 2023-07-06T21:39:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f2fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1ec0b81c0 0x7fd1ec152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f2fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1ec0ceda0 0x7fd1ec0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f2fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0c7d20 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f2fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0c8000 con 0x7fd1ec0ceda0 2023-07-06T21:39:39.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0c82e0 con 0x7fd1ec0b81c0 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.030+0000 7fd1f0d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51202/0 (socket says 172.21.15.84:51202) 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 -- 172.21.15.84:0/2195001455 learned_addr learned my addr 172.21.15.84:0/2195001455 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1ebfff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1ec0b81c0 0x7fd1ec152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f159c700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1ec0ceda0 0x7fd1ec0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 -- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1ec0b81c0 msgr2=0x7fd1ec152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1ec0b81c0 0x7fd1ec152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 -- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1ec0ceda0 msgr2=0x7fd1ec0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1ec0ceda0 0x7fd1ec0c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 -- 172.21.15.84:0/2195001455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d8009550 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f0d9b700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec153060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000e810 tx=0x7fd1e000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e000c070 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e0013610 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec0c85c0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.031+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00139c0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.032+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec0c8b80 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.033+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec04e1d0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.034+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd1e0013b20 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.035+0000 7fd1e97fa700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd1d006ace0 0x7fd1d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.035+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd1e00891f0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.035+0000 7fd1ebfff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd1d006ace0 0x7fd1d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.036+0000 7fd1ebfff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd1d006ace0 0x7fd1d006d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8014040 tx=0x7fd1d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.039+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd1e00542b0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.237+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd1ec0b4f30 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.237+0000 7fd1e97fa700 1 -- 172.21.15.84:0/2195001455 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd1e00578d0 con 0x7fd1ec0b1a80 2023-07-06T21:39:39.240 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:39:39.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd1d006ace0 msgr2=0x7fd1d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd1d006ace0 0x7fd1d006d190 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8014040 tx=0x7fd1d8008030 comp rx=0 tx=0).stop 2023-07-06T21:39:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 msgr2=0x7fd1ec153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec153060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000e810 tx=0x7fd1e000eb20 comp rx=0 tx=0).stop 2023-07-06T21:39:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 shutdown_connections 2023-07-06T21:39:39.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd1d006ace0 0x7fd1d006d190 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1ec0ceda0 0x7fd1ec0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd1ec0b1a80 0x7fd1ec153060 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 --2- 172.21.15.84:0/2195001455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1ec0b81c0 0x7fd1ec152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.239+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 >> 172.21.15.84:0/2195001455 conn(0x7fd1ec06b2b0 msgr2=0x7fd1ec0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.240+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 shutdown_connections 2023-07-06T21:39:39.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.240+0000 7fd1f2fff700 1 -- 172.21.15.84:0/2195001455 wait complete. 2023-07-06T21:39:39.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:39:39.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.452+0000 7f97d29ff700 1 Processor -- start 2023-07-06T21:39:39.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.452+0000 7f97d29ff700 1 -- start start 2023-07-06T21:39:39.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97d29ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97cc0b81c0 0x7f97cc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97d29ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97cc0b1a80 0x7f97cc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97d29ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97cc0d1830 con 0x7f97cc0ceda0 2023-07-06T21:39:39.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97d29ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc0cf330 con 0x7f97cc0b81c0 2023-07-06T21:39:39.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97d29ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97cc0d1270 con 0x7f97cc0b1a80 2023-07-06T21:39:39.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97cbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f97cc0b81c0 0x7f97cc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.453+0000 7f97d0f9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f97cc0ceda0 0x7f97cc15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47890/0 (socket says 172.21.15.84:47890) 2023-07-06T21:39:39.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97d0f9c700 1 -- 172.21.15.84:0/588041306 learned_addr learned my addr 172.21.15.84:0/588041306 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:39.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97cb7fe700 1 --2- 172.21.15.84:0/588041306 >> v2:172.21.15.107:3300/0 conn(0x7f97cc0b1a80 0x7f97cc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97cbfff700 1 -- 172.21.15.84:0/588041306 >> v2:172.21.15.107:3300/0 conn(0x7f97cc0b1a80 msgr2=0x7f97cc0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:39.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97cbfff700 1 --2- 172.21.15.84:0/588041306 >> v2:172.21.15.107:3300/0 conn(0x7f97cc0b1a80 0x7f97cc0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97cbfff700 1 -- 172.21.15.84:0/588041306 >> v1:172.21.15.84:6789/0 conn(0x7f97cc0ceda0 legacy=0x7f97cc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97cbfff700 1 -- 172.21.15.84:0/588041306 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc0d1550 con 0x7f97cc0b81c0 2023-07-06T21:39:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97caffd700 1 -- 172.21.15.84:0/588041306 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381091010 0 0) 0x7f97cc0d1830 con 0x7f97cc0ceda0 2023-07-06T21:39:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97cbfff700 1 --2- 172.21.15.84:0/588041306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0cf600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0d0770 tx=0x7f97bc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5bcc8cd838141b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97caffd700 1 -- 172.21.15.84:0/588041306 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011830 con 0x7f97cc0b81c0 2023-07-06T21:39:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97caffd700 1 -- 172.21.15.84:0/588041306 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc011990 con 0x7f97cc0b81c0 2023-07-06T21:39:39.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.454+0000 7f97caffd700 1 -- 172.21.15.84:0/588041306 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011d20 con 0x7f97cc0b81c0 2023-07-06T21:39:39.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 -- 172.21.15.84:0/588041306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 msgr2=0x7f97cc0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 --2- 172.21.15.84:0/588041306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0cf600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0d0770 tx=0x7f97bc009b20 comp rx=0 tx=0).stop 2023-07-06T21:39:39.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 -- 172.21.15.84:0/588041306 shutdown_connections 2023-07-06T21:39:39.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 --2- 172.21.15.84:0/588041306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0cf600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 --2- 172.21.15.84:0/588041306 >> v2:172.21.15.107:3300/0 conn(0x7f97cc0b1a80 0x7f97cc0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 -- 172.21.15.84:0/588041306 >> 172.21.15.84:0/588041306 conn(0x7f97cc06b2b0 msgr2=0x7f97cc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:39.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 -- 172.21.15.84:0/588041306 shutdown_connections 2023-07-06T21:39:39.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.455+0000 7f97d29ff700 1 -- 172.21.15.84:0/588041306 wait complete. 2023-07-06T21:39:39.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.457+0000 7f97d29ff700 1 Processor -- start 2023-07-06T21:39:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.457+0000 7f97d29ff700 1 -- start start 2023-07-06T21:39:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d29ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97cc0b1a80 0x7f97cc15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d29ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d29ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97cc0ceda0 0x7f97cc1e3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d29ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0ca4a0 con 0x7f97cc0b81c0 2023-07-06T21:39:39.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d29ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0ca6d0 con 0x7f97cc0ceda0 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d29ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc0ca9b0 con 0x7f97cc0b1a80 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51208/0 (socket says 172.21.15.84:51208) 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97cb7fe700 1 -- 172.21.15.84:0/4092338507 learned_addr learned my addr 172.21.15.84:0/4092338507 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97cbfff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97cc0b1a80 0x7f97cc15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97d0f9c700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97cc0ceda0 0x7f97cc1e3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.458+0000 7f97cb7fe700 1 -- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97cc0b1a80 msgr2=0x7f97cc15b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97cb7fe700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97cc0b1a80 0x7f97cc15b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97cb7fe700 1 -- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97cc0ceda0 msgr2=0x7f97cc1e3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97cb7fe700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97cc0ceda0 0x7f97cc1e3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97cb7fe700 1 -- 172.21.15.84:0/4092338507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0094a0 con 0x7f97cc0b81c0 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97cb7fe700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0c9f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f97c000e870 tx=0x7f97c000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0009590 con 0x7f97cc0b81c0 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c000ee70 con 0x7f97cc0b81c0 2023-07-06T21:39:39.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc0cac90 con 0x7f97cc0b81c0 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c000ca50 con 0x7f97cc0b81c0 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.459+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc0cb250 con 0x7f97cc0b81c0 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.460+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc04e1d0 con 0x7f97cc0b81c0 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.461+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97c000cbb0 con 0x7f97cc0b81c0 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.462+0000 7f97c8ff9700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97b806acb0 0x7f97b806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.462+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97c0089460 con 0x7f97cc0b81c0 2023-07-06T21:39:39.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.462+0000 7f97cbfff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97b806acb0 0x7f97b806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.463+0000 7f97cbfff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97b806acb0 0x7f97b806d160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0090b0 tx=0x7f97bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.466+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97c0057f90 con 0x7f97cc0b81c0 2023-07-06T21:39:39.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.663+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97cc0d01d0 con 0x7f97cc0b81c0 2023-07-06T21:39:39.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.663+0000 7f97c8ff9700 1 -- 172.21.15.84:0/4092338507 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97c000ce60 con 0x7f97cc0b81c0 2023-07-06T21:39:39.666 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97b806acb0 msgr2=0x7f97b806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97b806acb0 0x7f97b806d160 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0090b0 tx=0x7f97bc008030 comp rx=0 tx=0).stop 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 msgr2=0x7f97cc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0c9f60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f97c000e870 tx=0x7f97c000ec30 comp rx=0 tx=0).stop 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 shutdown_connections 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97b806acb0 0x7f97b806d160 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97cc0ceda0 0x7f97cc1e3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97cc0b1a80 0x7f97cc15b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.665+0000 7f97d29ff700 1 --2- 172.21.15.84:0/4092338507 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97cc0b81c0 0x7f97cc0c9f60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.666+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 >> 172.21.15.84:0/4092338507 conn(0x7f97cc06b2b0 msgr2=0x7f97cc0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.666+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 shutdown_connections 2023-07-06T21:39:39.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.666+0000 7f97d29ff700 1 -- 172.21.15.84:0/4092338507 wait complete. 2023-07-06T21:39:39.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:39:39.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.878+0000 7f15b99ff700 1 Processor -- start 2023-07-06T21:39:39.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.879+0000 7f15b99ff700 1 -- start start 2023-07-06T21:39:39.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b99ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40cfd10 0x7f15b40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b99ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15b40b1a80 0x7f15b40d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b99ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b40d1830 con 0x7f15b40b81c0 2023-07-06T21:39:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b99ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b40cf330 con 0x7f15b40b1a80 2023-07-06T21:39:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b99ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40d1270 con 0x7f15b40cfd10 2023-07-06T21:39:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b37fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15b40b1a80 0x7f15b40d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b37fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15b40b1a80 0x7f15b40d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51222/0 (socket says 172.21.15.84:51222) 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b37fe700 1 -- 172.21.15.84:0/612840209 learned_addr learned my addr 172.21.15.84:0/612840209 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b27fc700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40cfd10 0x7f15b40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b37fe700 1 -- 172.21.15.84:0/612840209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40cfd10 msgr2=0x7f15b40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b37fe700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40cfd10 0x7f15b40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.880+0000 7f15b37fe700 1 -- 172.21.15.84:0/612840209 >> v1:172.21.15.107:6789/0 conn(0x7f15b40b81c0 legacy=0x7f15b40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.881+0000 7f15b37fe700 1 -- 172.21.15.84:0/612840209 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40d1550 con 0x7f15b40b1a80 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.881+0000 7f15b27fc700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40cfd10 0x7f15b40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:39.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.881+0000 7f15b37fe700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40b1a80 0x7f15b40d28c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009e80 tx=0x7f15ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91645db362f85111 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.881+0000 7f15b1ffb700 1 -- 172.21.15.84:0/612840209 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00e070 con 0x7f15b40b1a80 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.881+0000 7f15b1ffb700 1 -- 172.21.15.84:0/612840209 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac00b980 con 0x7f15b40b1a80 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.881+0000 7f15b1ffb700 1 -- 172.21.15.84:0/612840209 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac00bd30 con 0x7f15b40b1a80 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 -- 172.21.15.84:0/612840209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40b1a80 msgr2=0x7f15b40d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40b1a80 0x7f15b40d28c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009e80 tx=0x7f15ac00b330 comp rx=0 tx=0).stop 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 -- 172.21.15.84:0/612840209 shutdown_connections 2023-07-06T21:39:39.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40b1a80 0x7f15b40d28c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 --2- 172.21.15.84:0/612840209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40cfd10 0x7f15b40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 -- 172.21.15.84:0/612840209 >> 172.21.15.84:0/612840209 conn(0x7f15b406b2b0 msgr2=0x7f15b406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:39.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 -- 172.21.15.84:0/612840209 shutdown_connections 2023-07-06T21:39:39.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.882+0000 7f15b99ff700 1 -- 172.21.15.84:0/612840209 wait complete. 2023-07-06T21:39:39.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.883+0000 7f15b99ff700 1 Processor -- start 2023-07-06T21:39:39.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 -- start start 2023-07-06T21:39:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b40b1a80 0x7f15b40c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40b81c0 0x7f15b41ef300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 0x7f15b41f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40c7700 con 0x7f15b40cfd10 2023-07-06T21:39:39.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b4159bf0 con 0x7f15b40b1a80 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.884+0000 7f15b99ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b40c6720 con 0x7f15b40b81c0 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 0x7f15b41f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 0x7f15b41f1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51226/0 (socket says 172.21.15.84:51226) 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 -- 172.21.15.84:0/2189280732 learned_addr learned my addr 172.21.15.84:0/2189280732 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b2ffd700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b40b1a80 0x7f15b40c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b27fc700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40b81c0 0x7f15b41ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 -- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40b81c0 msgr2=0x7f15b41ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40b81c0 0x7f15b41ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 -- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b40b1a80 msgr2=0x7f15b40c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b40b1a80 0x7f15b40c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 -- 172.21.15.84:0/2189280732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac009b30 con 0x7f15b40cfd10 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b27fc700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40b81c0 0x7f15b41ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.885+0000 7f15b37fe700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 0x7f15b41f1e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f15ac012000 tx=0x7f15ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.886+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac004960 con 0x7f15b40cfd10 2023-07-06T21:39:39.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.886+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac0092e0 con 0x7f15b40cfd10 2023-07-06T21:39:39.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.886+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac01c660 con 0x7f15b40cfd10 2023-07-06T21:39:39.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.886+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b41f2a40 con 0x7f15b40cfd10 2023-07-06T21:39:39.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.886+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b40c69e0 con 0x7f15b40cfd10 2023-07-06T21:39:39.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.887+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b40666c0 con 0x7f15b40cfd10 2023-07-06T21:39:39.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.888+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f15ac01c7c0 con 0x7f15b40cfd10 2023-07-06T21:39:39.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.889+0000 7f159bfff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f15a006ace0 0x7f15a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:39.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.889+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15ac08af00 con 0x7f15b40cfd10 2023-07-06T21:39:39.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.889+0000 7f15b2ffd700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f15a006ace0 0x7f15a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:39.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.890+0000 7f15b2ffd700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f15a006ace0 0x7f15a006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f15a4000f60 tx=0x7f15a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:39.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:39.893+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f15ac055fa0 con 0x7f15b40cfd10 2023-07-06T21:39:40.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.089+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15b404e1d0 con 0x7f15b40cfd10 2023-07-06T21:39:40.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.090+0000 7f159bfff700 1 -- 172.21.15.84:0/2189280732 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15ac0595c0 con 0x7f15b40cfd10 2023-07-06T21:39:40.093 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:39:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f15a006ace0 msgr2=0x7f15a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f15a006ace0 0x7f15a006d190 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f15a4000f60 tx=0x7f15a4009040 comp rx=0 tx=0).stop 2023-07-06T21:39:40.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 msgr2=0x7f15b41f1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 0x7f15b41f1e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f15ac012000 tx=0x7f15ac004ad0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 shutdown_connections 2023-07-06T21:39:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f15a006ace0 0x7f15a006d190 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15b40cfd10 0x7f15b41f1e80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b40b1a80 0x7f15b40c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 --2- 172.21.15.84:0/2189280732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15b40b81c0 0x7f15b41ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 >> 172.21.15.84:0/2189280732 conn(0x7f15b406b2b0 msgr2=0x7f15b40d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 shutdown_connections 2023-07-06T21:39:40.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.092+0000 7f15b99ff700 1 -- 172.21.15.84:0/2189280732 wait complete. 2023-07-06T21:39:40.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:39:40.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.303+0000 7f932e1ff700 1 Processor -- start 2023-07-06T21:39:40.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.304+0000 7f932e1ff700 1 -- start start 2023-07-06T21:39:40.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f932e1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93280b83c0 0x7f93280cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f932e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280d3710 0x7f93280cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f932e1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280cfee0 con 0x7f93280cfa90 2023-07-06T21:39:40.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f932e1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280d0a80 con 0x7f93280b83c0 2023-07-06T21:39:40.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f932e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280b8790 con 0x7f93280d3710 2023-07-06T21:39:40.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93280b83c0 0x7f93280cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93280b83c0 0x7f93280cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51238/0 (socket says 172.21.15.84:51238) 2023-07-06T21:39:40.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 -- 172.21.15.84:0/1784969182 learned_addr learned my addr 172.21.15.84:0/1784969182 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:40.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f931ffff700 1 --2- 172.21.15.84:0/1784969182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280d3710 0x7f93280cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 -- 172.21.15.84:0/1784969182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280d3710 msgr2=0x7f93280cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:40.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 --2- 172.21.15.84:0/1784969182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280d3710 0x7f93280cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 -- 172.21.15.84:0/1784969182 >> v1:172.21.15.107:6789/0 conn(0x7f93280cfa90 legacy=0x7f932815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.305+0000 7f93277fe700 1 -- 172.21.15.84:0/1784969182 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93280b4e70 con 0x7f93280b83c0 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.306+0000 7f93277fe700 1 --2- 172.21.15.84:0/1784969182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280b83c0 0x7f93280cd200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93180098f0 tx=0x7f9318009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b960916f0058a26d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.306+0000 7f9326ffd700 1 -- 172.21.15.84:0/1784969182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800b8a0 con 0x7f93280b83c0 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.306+0000 7f9326ffd700 1 -- 172.21.15.84:0/1784969182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931800ba00 con 0x7f93280b83c0 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.306+0000 7f9326ffd700 1 -- 172.21.15.84:0/1784969182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800bd90 con 0x7f93280b83c0 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.306+0000 7f932e1ff700 1 -- 172.21.15.84:0/1784969182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280b83c0 msgr2=0x7f93280cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.306+0000 7f932e1ff700 1 --2- 172.21.15.84:0/1784969182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280b83c0 0x7f93280cd200 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f93180098f0 tx=0x7f9318009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.307+0000 7f932e1ff700 1 -- 172.21.15.84:0/1784969182 shutdown_connections 2023-07-06T21:39:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.307+0000 7f932e1ff700 1 --2- 172.21.15.84:0/1784969182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280b83c0 0x7f93280cd200 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.307+0000 7f932e1ff700 1 --2- 172.21.15.84:0/1784969182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280d3710 0x7f93280cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.307+0000 7f932e1ff700 1 -- 172.21.15.84:0/1784969182 >> 172.21.15.84:0/1784969182 conn(0x7f932806b350 msgr2=0x7f932806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.307+0000 7f932e1ff700 1 -- 172.21.15.84:0/1784969182 shutdown_connections 2023-07-06T21:39:40.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.307+0000 7f932e1ff700 1 -- 172.21.15.84:0/1784969182 wait complete. 2023-07-06T21:39:40.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.308+0000 7f932e1ff700 1 Processor -- start 2023-07-06T21:39:40.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 -- start start 2023-07-06T21:39:40.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280b83c0 0x7f9328153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 0x7f9328153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93280d3710 0x7f93280c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280c6670 con 0x7f93280cfa90 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280c9fb0 con 0x7f93280d3710 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.309+0000 7f932e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cac80 con 0x7f93280b83c0 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 0x7f9328153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 0x7f9328153e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51252/0 (socket says 172.21.15.84:51252) 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 -- 172.21.15.84:0/3305482147 learned_addr learned my addr 172.21.15.84:0/3305482147 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f93277fe700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280b83c0 0x7f9328153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f9327fff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93280d3710 0x7f93280c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 -- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280b83c0 msgr2=0x7f9328153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280b83c0 0x7f9328153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 -- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93280d3710 msgr2=0x7f93280c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93280d3710 0x7f93280c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 -- 172.21.15.84:0/3305482147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9318009550 con 0x7f93280cfa90 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f9327fff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93280d3710 0x7f93280c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:40.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.310+0000 7f931ffff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 0x7f9328153e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f931000d830 tx=0x7f931000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:40.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.311+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93100125f0 con 0x7f93280cfa90 2023-07-06T21:39:40.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.311+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9310012750 con 0x7f93280cfa90 2023-07-06T21:39:40.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.311+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280caf60 con 0x7f93280cfa90 2023-07-06T21:39:40.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.311+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310012ae0 con 0x7f93280cfa90 2023-07-06T21:39:40.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.311+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93280c68c0 con 0x7f93280cfa90 2023-07-06T21:39:40.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.312+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93280c98d0 con 0x7f93280cfa90 2023-07-06T21:39:40.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.313+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9310012c40 con 0x7f93280cfa90 2023-07-06T21:39:40.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.314+0000 7f9324ff9700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f930806ad10 0x7f930806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.314+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93100895a0 con 0x7f93280cfa90 2023-07-06T21:39:40.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.314+0000 7f93277fe700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f930806ad10 0x7f930806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.315+0000 7f93277fe700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f930806ad10 0x7f930806d1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9318014040 tx=0x7f9318008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:40.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.318+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9310054660 con 0x7f93280cfa90 2023-07-06T21:39:40.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.555+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93280cf550 con 0x7f93280cfa90 2023-07-06T21:39:40.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.555+0000 7f9324ff9700 1 -- 172.21.15.84:0/3305482147 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f9310057c80 con 0x7f93280cfa90 2023-07-06T21:39:40.561 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:40.561 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:39:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f930806ad10 msgr2=0x7f930806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f930806ad10 0x7f930806d1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9318014040 tx=0x7f9318008030 comp rx=0 tx=0).stop 2023-07-06T21:39:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 msgr2=0x7f9328153e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 0x7f9328153e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f931000d830 tx=0x7f931000db40 comp rx=0 tx=0).stop 2023-07-06T21:39:40.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 shutdown_connections 2023-07-06T21:39:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f930806ad10 0x7f930806d1c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93280d3710 0x7f93280c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93280b83c0 0x7f9328153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 --2- 172.21.15.84:0/3305482147 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93280cfa90 0x7f9328153e70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 >> 172.21.15.84:0/3305482147 conn(0x7f932806b350 msgr2=0x7f932815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 shutdown_connections 2023-07-06T21:39:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.558+0000 7f932e1ff700 1 -- 172.21.15.84:0/3305482147 wait complete. 2023-07-06T21:39:40.566 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:39:40.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:40.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.771+0000 7fe6409ff700 1 Processor -- start 2023-07-06T21:39:40.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.771+0000 7fe6409ff700 1 -- start start 2023-07-06T21:39:40.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.772+0000 7fe6409ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe63c0cfc60 0x7fe63c167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.772+0000 7fe6409ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0cd910 con 0x7fe63c0d3710 2023-07-06T21:39:40.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe6409ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe63c0d0a80 con 0x7fe63c0b83c0 2023-07-06T21:39:40.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe6409ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0b8790 con 0x7fe63c0cfc60 2023-07-06T21:39:40.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe633fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe63c0d3710 0x7fe63c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47900/0 (socket says 172.21.15.84:47900) 2023-07-06T21:39:40.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe633fff700 1 -- 172.21.15.84:0/78700409 learned_addr learned my addr 172.21.15.84:0/78700409 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:40.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe63ad9d700 1 --2- 172.21.15.84:0/78700409 >> v2:172.21.15.107:3300/0 conn(0x7fe63c0cfc60 0x7fe63c167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372196119 0 0) 0x7fe624001110 con 0x7fe63c0d3710 2023-07-06T21:39:40.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6280035b0 con 0x7fe63c0d3710 2023-07-06T21:39:40.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975296636 0 0) 0x7fe63c0d0a80 con 0x7fe63c0b83c0 2023-07-06T21:39:40.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe624001110 con 0x7fe63c0b83c0 2023-07-06T21:39:40.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196590301 0 0) 0x7fe6280035b0 con 0x7fe63c0d3710 2023-07-06T21:39:40.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.773+0000 7fe63ad9d700 1 -- 172.21.15.84:0/78700409 >> v1:172.21.15.107:6789/0 conn(0x7fe63c0b83c0 legacy=0x7fe63c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe63ad9d700 1 -- 172.21.15.84:0/78700409 >> v1:172.21.15.84:6789/0 conn(0x7fe63c0d3710 legacy=0x7fe63c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe63ad9d700 1 -- 172.21.15.84:0/78700409 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0b4e70 con 0x7fe63c0cfc60 2023-07-06T21:39:40.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe63ad9d700 1 --2- 172.21.15.84:0/78700409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c167010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe634009d90 tx=0x7fe63400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80d301e836fa94bf server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:40.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63400e070 con 0x7fe63c0cfc60 2023-07-06T21:39:40.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe634003a20 con 0x7fe63c0cfc60 2023-07-06T21:39:40.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe639d9b700 1 -- 172.21.15.84:0/78700409 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624001800 con 0x7fe63c0cfc60 2023-07-06T21:39:40.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.774+0000 7fe6409ff700 1 -- 172.21.15.84:0/78700409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 msgr2=0x7fe63c167010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.775+0000 7fe6409ff700 1 --2- 172.21.15.84:0/78700409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c167010 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe634009d90 tx=0x7fe63400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:40.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.775+0000 7fe6409ff700 1 -- 172.21.15.84:0/78700409 shutdown_connections 2023-07-06T21:39:40.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.775+0000 7fe6409ff700 1 --2- 172.21.15.84:0/78700409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c167010 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.775+0000 7fe6409ff700 1 -- 172.21.15.84:0/78700409 >> 172.21.15.84:0/78700409 conn(0x7fe63c06b350 msgr2=0x7fe63c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:40.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.775+0000 7fe6409ff700 1 -- 172.21.15.84:0/78700409 shutdown_connections 2023-07-06T21:39:40.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.775+0000 7fe6409ff700 1 -- 172.21.15.84:0/78700409 wait complete. 2023-07-06T21:39:40.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.776+0000 7fe6409ff700 1 Processor -- start 2023-07-06T21:39:40.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.776+0000 7fe6409ff700 1 -- start start 2023-07-06T21:39:40.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.777+0000 7fe6409ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 0x7fe63c1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.777+0000 7fe6409ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.777+0000 7fe6409ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3710 0x7fe63c0b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.777+0000 7fe6409ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1f4f20 con 0x7fe63c0b83c0 2023-07-06T21:39:40.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.777+0000 7fe6409ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c15da70 con 0x7fe63c0cfc60 2023-07-06T21:39:40.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.777+0000 7fe6409ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c15dd50 con 0x7fe63c0d3710 2023-07-06T21:39:40.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 0x7fe63c1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 0x7fe63c1f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51268/0 (socket says 172.21.15.84:51268) 2023-07-06T21:39:40.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 -- 172.21.15.84:0/2932805447 learned_addr learned my addr 172.21.15.84:0/2932805447 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:40.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe633fff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63ad9d700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3710 0x7fe63c0b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 -- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3710 msgr2=0x7fe63c0b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3710 0x7fe63c0b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 -- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 msgr2=0x7fe63c1f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c1f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 -- 172.21.15.84:0/2932805447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe634009a70 con 0x7fe63c0b83c0 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe633fff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c1f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe63a59c700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 0x7fe63c1f1c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c500 tx=0x7fe62c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0052d0 con 0x7fe63c0b83c0 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c005450 con 0x7fe63c0b83c0 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c0c77d0 con 0x7fe63c0b83c0 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.778+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c0038c0 con 0x7fe63c0b83c0 2023-07-06T21:39:40.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.779+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c0c7d90 con 0x7fe63c0b83c0 2023-07-06T21:39:40.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.779+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c0c7fb0 con 0x7fe63c0b83c0 2023-07-06T21:39:40.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.781+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe62c003a20 con 0x7fe63c0b83c0 2023-07-06T21:39:40.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.781+0000 7fe632ffd700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe61c06ace0 0x7fe61c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:40.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.782+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe62c088910 con 0x7fe63c0b83c0 2023-07-06T21:39:40.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.782+0000 7fe633fff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe61c06ace0 0x7fe61c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:40.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.782+0000 7fe633fff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe61c06ace0 0x7fe61c06d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe624001eb0 tx=0x7fe624001ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:40.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.786+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe62c0539b0 con 0x7fe63c0b83c0 2023-07-06T21:39:40.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.984+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe63c15df10 con 0x7fe63c0b83c0 2023-07-06T21:39:40.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.985+0000 7fe632ffd700 1 -- 172.21.15.84:0/2932805447 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7fe62c056fd0 con 0x7fe63c0b83c0 2023-07-06T21:39:40.989 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:40.989 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe61c06ace0 msgr2=0x7fe61c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe61c06ace0 0x7fe61c06d190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe624001eb0 tx=0x7fe624001ca0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 msgr2=0x7fe63c1f1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 0x7fe63c1f1c10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c500 tx=0x7fe62c00c810 comp rx=0 tx=0).stop 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 shutdown_connections 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe61c06ace0 0x7fe61c06d190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3710 0x7fe63c0b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe63c0b83c0 0x7fe63c1f1c10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 --2- 172.21.15.84:0/2932805447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c0cfc60 0x7fe63c1f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:40.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 >> 172.21.15.84:0/2932805447 conn(0x7fe63c06b350 msgr2=0x7fe63c0b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:40.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 shutdown_connections 2023-07-06T21:39:40.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:40.988+0000 7fe6409ff700 1 -- 172.21.15.84:0/2932805447 wait complete. 2023-07-06T21:39:41.009 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:39:41.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:39:41.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.203+0000 7f8a69fff700 1 Processor -- start 2023-07-06T21:39:41.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.203+0000 7f8a69fff700 1 -- start start 2023-07-06T21:39:41.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.204+0000 7f8a69fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a640b81c0 0x7f8a640cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.204+0000 7f8a69fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640b1a80 0x7f8a640cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.204+0000 7f8a69fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a640d1830 con 0x7f8a640ceda0 2023-07-06T21:39:41.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.204+0000 7f8a69fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a640cf330 con 0x7f8a640b81c0 2023-07-06T21:39:41.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.204+0000 7f8a69fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640d1270 con 0x7f8a640b1a80 2023-07-06T21:39:41.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a640b81c0 0x7f8a640cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8a640b81c0 0x7f8a640cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51278/0 (socket says 172.21.15.84:51278) 2023-07-06T21:39:41.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 -- 172.21.15.84:0/171181270 learned_addr learned my addr 172.21.15.84:0/171181270 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:41.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a62ffd700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640b1a80 0x7f8a640cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 -- 172.21.15.84:0/171181270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640b1a80 msgr2=0x7f8a640cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:41.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640b1a80 0x7f8a640cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 -- 172.21.15.84:0/171181270 >> v1:172.21.15.107:6789/0 conn(0x7f8a640ceda0 legacy=0x7f8a6415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 -- 172.21.15.84:0/171181270 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640d1550 con 0x7f8a640b81c0 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a62ffd700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640b1a80 0x7f8a640cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a637fe700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a640cf600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098f0 tx=0x7f8a54009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95376d2a7f1d1d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.205+0000 7f8a627fc700 1 -- 172.21.15.84:0/171181270 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400b8a0 con 0x7f8a640b81c0 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a627fc700 1 -- 172.21.15.84:0/171181270 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5400ba00 con 0x7f8a640b81c0 2023-07-06T21:39:41.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a627fc700 1 -- 172.21.15.84:0/171181270 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400bd90 con 0x7f8a640b81c0 2023-07-06T21:39:41.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 -- 172.21.15.84:0/171181270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 msgr2=0x7f8a640cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a640cf600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8a540098f0 tx=0x7f8a54009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:41.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 -- 172.21.15.84:0/171181270 shutdown_connections 2023-07-06T21:39:41.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a640cf600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 --2- 172.21.15.84:0/171181270 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640b1a80 0x7f8a640cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 -- 172.21.15.84:0/171181270 >> 172.21.15.84:0/171181270 conn(0x7f8a6406b2b0 msgr2=0x7f8a6406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:41.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 -- 172.21.15.84:0/171181270 shutdown_connections 2023-07-06T21:39:41.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.206+0000 7f8a69fff700 1 -- 172.21.15.84:0/171181270 wait complete. 2023-07-06T21:39:41.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.208+0000 7f8a69fff700 1 Processor -- start 2023-07-06T21:39:41.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.208+0000 7f8a69fff700 1 -- start start 2023-07-06T21:39:41.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a69fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a640b1a80 0x7f8a6415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a69fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a6415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a69fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640ceda0 0x7f8a641e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a69fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c9ab0 con 0x7f8a640b81c0 2023-07-06T21:39:41.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a69fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c7920 con 0x7f8a640b1a80 2023-07-06T21:39:41.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a69fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6415acc0 con 0x7f8a640ceda0 2023-07-06T21:39:41.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a62ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a6415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a62ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a6415a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51284/0 (socket says 172.21.15.84:51284) 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a62ffd700 1 -- 172.21.15.84:0/2493001598 learned_addr learned my addr 172.21.15.84:0/2493001598 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a637fe700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a640b1a80 0x7f8a6415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.209+0000 7f8a63fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640ceda0 0x7f8a641e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a62ffd700 1 -- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640ceda0 msgr2=0x7f8a641e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a62ffd700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640ceda0 0x7f8a641e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a62ffd700 1 -- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a640b1a80 msgr2=0x7f8a6415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a62ffd700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a640b1a80 0x7f8a6415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a62ffd700 1 -- 172.21.15.84:0/2493001598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a54009550 con 0x7f8a640b81c0 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a63fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640ceda0 0x7f8a641e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a62ffd700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a6415a780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800e840 tx=0x7f8a5800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58013610 con 0x7f8a640b81c0 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58013770 con 0x7f8a640b81c0 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640c97d0 con 0x7f8a640b81c0 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58013b20 con 0x7f8a640b81c0 2023-07-06T21:39:41.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.210+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a640c7380 con 0x7f8a640b81c0 2023-07-06T21:39:41.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.211+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a640c7ad0 con 0x7f8a640b81c0 2023-07-06T21:39:41.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.212+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8a58013c80 con 0x7f8a640b81c0 2023-07-06T21:39:41.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.213+0000 7f8a4bfff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a5006ace0 0x7f8a5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.213+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8a58089150 con 0x7f8a640b81c0 2023-07-06T21:39:41.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.213+0000 7f8a637fe700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a5006ace0 0x7f8a5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.214+0000 7f8a637fe700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a5006ace0 0x7f8a5006d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009c40 tx=0x7f8a54008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:41.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.217+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8a640c7ad0 con 0x7f8a640b81c0 2023-07-06T21:39:41.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.408+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a640c7ad0 con 0x7f8a640b81c0 2023-07-06T21:39:41.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.409+0000 7f8a4bfff700 1 -- 172.21.15.84:0/2493001598 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a640c7ad0 con 0x7f8a640b81c0 2023-07-06T21:39:41.412 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:39:41.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a5006ace0 msgr2=0x7f8a5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a5006ace0 0x7f8a5006d190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009c40 tx=0x7f8a54008030 comp rx=0 tx=0).stop 2023-07-06T21:39:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 msgr2=0x7f8a6415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a6415a780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800e840 tx=0x7f8a5800ec00 comp rx=0 tx=0).stop 2023-07-06T21:39:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 shutdown_connections 2023-07-06T21:39:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a5006ace0 0x7f8a5006d190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a640ceda0 0x7f8a641e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a640b1a80 0x7f8a6415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 --2- 172.21.15.84:0/2493001598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8a640b81c0 0x7f8a6415a780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 >> 172.21.15.84:0/2493001598 conn(0x7f8a6406b2b0 msgr2=0x7f8a640cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 shutdown_connections 2023-07-06T21:39:41.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.411+0000 7f8a69fff700 1 -- 172.21.15.84:0/2493001598 wait complete. 2023-07-06T21:39:41.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:39:41.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.624+0000 7f150dbff700 1 Processor -- start 2023-07-06T21:39:41.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.624+0000 7f150dbff700 1 -- start start 2023-07-06T21:39:41.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.625+0000 7f150dbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1508069e90 0x7f150806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.625+0000 7f150dbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f150806dde0 0x7f150806a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.625+0000 7f150dbff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080c94b0 con 0x7f15080cf280 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.625+0000 7f150dbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150806d350 con 0x7f150806dde0 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.625+0000 7f150dbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150806a7a0 con 0x7f1508069e90 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1506ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f150806dde0 0x7f150806a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1507fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f15080cf280 0x7f15080b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:47906/0 (socket says 172.21.15.84:47906) 2023-07-06T21:39:41.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1507fff700 1 -- 172.21.15.84:0/3873079712 learned_addr learned my addr 172.21.15.84:0/3873079712 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f15077fe700 1 --2- 172.21.15.84:0/3873079712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1508069e90 0x7f150806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1506ffd700 1 -- 172.21.15.84:0/3873079712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1508069e90 msgr2=0x7f150806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1506ffd700 1 --2- 172.21.15.84:0/3873079712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1508069e90 0x7f150806ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1506ffd700 1 -- 172.21.15.84:0/3873079712 >> v1:172.21.15.84:6789/0 conn(0x7f15080cf280 legacy=0x7f15080b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1506ffd700 1 -- 172.21.15.84:0/3873079712 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080ca9b0 con 0x7f150806dde0 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f15067fc700 1 -- 172.21.15.84:0/3873079712 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1572146732 0 0) 0x7f15080c94b0 con 0x7f15080cf280 2023-07-06T21:39:41.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f1506ffd700 1 --2- 172.21.15.84:0/3873079712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f150806a260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009840 tx=0x7f14fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b41d3b2d05c8ad3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:41.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.626+0000 7f15067fc700 1 -- 172.21.15.84:0/3873079712 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011830 con 0x7f150806dde0 2023-07-06T21:39:41.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f15067fc700 1 -- 172.21.15.84:0/3873079712 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc011990 con 0x7f150806dde0 2023-07-06T21:39:41.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f15067fc700 1 -- 172.21.15.84:0/3873079712 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc011d20 con 0x7f150806dde0 2023-07-06T21:39:41.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 -- 172.21.15.84:0/3873079712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 msgr2=0x7f150806a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 --2- 172.21.15.84:0/3873079712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f150806a260 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f14fc009840 tx=0x7f14fc009b50 comp rx=0 tx=0).stop 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 -- 172.21.15.84:0/3873079712 shutdown_connections 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 --2- 172.21.15.84:0/3873079712 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1508069e90 0x7f150806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 --2- 172.21.15.84:0/3873079712 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f150806a260 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 -- 172.21.15.84:0/3873079712 >> 172.21.15.84:0/3873079712 conn(0x7f15080cbf10 msgr2=0x7f15080ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 -- 172.21.15.84:0/3873079712 shutdown_connections 2023-07-06T21:39:41.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.627+0000 7f150dbff700 1 -- 172.21.15.84:0/3873079712 wait complete. 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.629+0000 7f150dbff700 1 Processor -- start 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.629+0000 7f150dbff700 1 -- start start 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f150dbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1508069e90 0x7f15081f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f150dbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f15081ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f150dbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15080cf280 0x7f150815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f150dbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150815bc80 con 0x7f150806dde0 2023-07-06T21:39:41.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f150dbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150815cae0 con 0x7f1508069e90 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f150dbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15081f0d30 con 0x7f15080cf280 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f15081ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f15081ed0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51302/0 (socket says 172.21.15.84:51302) 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 -- 172.21.15.84:0/3544175262 learned_addr learned my addr 172.21.15.84:0/3544175262 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1507fff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15080cf280 0x7f150815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f15077fe700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1508069e90 0x7f15081f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 -- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15080cf280 msgr2=0x7f150815cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15080cf280 0x7f150815cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 -- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1508069e90 msgr2=0x7f15081f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1508069e90 0x7f15081f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1506ffd700 1 -- 172.21.15.84:0/3544175262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc0094a0 con 0x7f150806dde0 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f1507fff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15080cf280 0x7f150815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.630+0000 7f15077fe700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1508069e90 0x7f15081f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.631+0000 7f1506ffd700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f15081ed0f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0090b0 tx=0x7f14fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.631+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc0038a0 con 0x7f150806dde0 2023-07-06T21:39:41.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.631+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc01e070 con 0x7f150806dde0 2023-07-06T21:39:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.631+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150815bed0 con 0x7f150806dde0 2023-07-06T21:39:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.631+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc003c00 con 0x7f150806dde0 2023-07-06T21:39:41.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.631+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150815c490 con 0x7f150806dde0 2023-07-06T21:39:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.632+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150815cc90 con 0x7f150806dde0 2023-07-06T21:39:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.633+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f14fc01a420 con 0x7f150806dde0 2023-07-06T21:39:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.634+0000 7f14effff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f14f406ad30 0x7f14f406d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.634+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14fc08a5b0 con 0x7f150806dde0 2023-07-06T21:39:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.634+0000 7f15077fe700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f14f406ad30 0x7f14f406d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:41.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.635+0000 7f15077fe700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f14f406ad30 0x7f14f406d1e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1508153a90 tx=0x7f14f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:41.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.638+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f150815cc90 con 0x7f150806dde0 2023-07-06T21:39:41.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.833+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f150815cc90 con 0x7f150806dde0 2023-07-06T21:39:41.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.833+0000 7f14effff700 1 -- 172.21.15.84:0/3544175262 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f150815cc90 con 0x7f150806dde0 2023-07-06T21:39:41.837 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:39:41.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f14f406ad30 msgr2=0x7f14f406d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f14f406ad30 0x7f14f406d1e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1508153a90 tx=0x7f14f8008040 comp rx=0 tx=0).stop 2023-07-06T21:39:41.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 msgr2=0x7f15081ed0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:41.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f15081ed0f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14fc0090b0 tx=0x7f14fc004750 comp rx=0 tx=0).stop 2023-07-06T21:39:41.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 shutdown_connections 2023-07-06T21:39:41.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f14f406ad30 0x7f14f406d1e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15080cf280 0x7f150815cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1508069e90 0x7f15081f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 --2- 172.21.15.84:0/3544175262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f150806dde0 0x7f15081ed0f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:41.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.835+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 >> 172.21.15.84:0/3544175262 conn(0x7f15080cbf10 msgr2=0x7f15080b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:41.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.836+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 shutdown_connections 2023-07-06T21:39:41.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:41.836+0000 7f150dbff700 1 -- 172.21.15.84:0/3544175262 wait complete. 2023-07-06T21:39:41.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:39:42.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.053+0000 7fc6e3fff700 1 Processor -- start 2023-07-06T21:39:42.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.053+0000 7fc6e3fff700 1 -- start start 2023-07-06T21:39:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.054+0000 7fc6e3fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc6dc0b81c0 0x7fc6dc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.054+0000 7fc6e3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6dc0b1a80 0x7fc6dc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.054+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0ceda0 0x7fc6dc15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.054+0000 7fc6e3fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc0d1830 con 0x7fc6dc0b1a80 2023-07-06T21:39:42.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e3fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc0cf330 con 0x7fc6dc0b81c0 2023-07-06T21:39:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e3fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0d1270 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6dc0b1a80 0x7fc6dc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc6dc0b1a80 0x7fc6dc0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51312/0 (socket says 172.21.15.84:51312) 2023-07-06T21:39:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 -- 172.21.15.84:0/795746071 learned_addr learned my addr 172.21.15.84:0/795746071 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e259c700 1 --2- 172.21.15.84:0/795746071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0ceda0 0x7fc6dc15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e1d9b700 1 --2- 172.21.15.84:0/795746071 >> v2:172.21.15.107:3300/0 conn(0x7fc6dc0b81c0 0x7fc6dc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 -- 172.21.15.84:0/795746071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0ceda0 msgr2=0x7fc6dc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:42.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 --2- 172.21.15.84:0/795746071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0ceda0 0x7fc6dc15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 -- 172.21.15.84:0/795746071 >> v2:172.21.15.107:3300/0 conn(0x7fc6dc0b81c0 msgr2=0x7fc6dc0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:42.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 --2- 172.21.15.84:0/795746071 >> v2:172.21.15.107:3300/0 conn(0x7fc6dc0b81c0 0x7fc6dc0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 -- 172.21.15.84:0/795746071 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0d1550 con 0x7fc6dc0b1a80 2023-07-06T21:39:42.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.055+0000 7fc6e159a700 1 --2- 172.21.15.84:0/795746071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc0cfb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0b2620 tx=0x7fc6cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b66824ba04f4e33 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.057+0000 7fc6e0d99700 1 -- 172.21.15.84:0/795746071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc011830 con 0x7fc6dc0b1a80 2023-07-06T21:39:42.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e0d99700 1 -- 172.21.15.84:0/795746071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc011990 con 0x7fc6dc0b1a80 2023-07-06T21:39:42.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e0d99700 1 -- 172.21.15.84:0/795746071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc011d20 con 0x7fc6dc0b1a80 2023-07-06T21:39:42.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 -- 172.21.15.84:0/795746071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0b1a80 msgr2=0x7fc6dc0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/795746071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc0cfb40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0b2620 tx=0x7fc6cc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 -- 172.21.15.84:0/795746071 shutdown_connections 2023-07-06T21:39:42.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/795746071 >> v2:172.21.15.107:3300/0 conn(0x7fc6dc0b81c0 0x7fc6dc0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/795746071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0ceda0 0x7fc6dc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/795746071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0b1a80 0x7fc6dc0cfb40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 -- 172.21.15.84:0/795746071 >> 172.21.15.84:0/795746071 conn(0x7fc6dc06b2b0 msgr2=0x7fc6dc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:42.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 -- 172.21.15.84:0/795746071 shutdown_connections 2023-07-06T21:39:42.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.058+0000 7fc6e3fff700 1 -- 172.21.15.84:0/795746071 wait complete. 2023-07-06T21:39:42.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.060+0000 7fc6e3fff700 1 Processor -- start 2023-07-06T21:39:42.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.060+0000 7fc6e3fff700 1 -- start start 2023-07-06T21:39:42.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0b1a80 0x7fc6dc1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6dc0b81c0 0x7fc6dc153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 0x7fc6dc0b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e3fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc153720 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e3fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0cb140 con 0x7fc6dc0b81c0 2023-07-06T21:39:42.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e3fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0cb420 con 0x7fc6dc0b1a80 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 0x7fc6dc0b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 0x7fc6dc0b7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51318/0 (socket says 172.21.15.84:51318) 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e259c700 1 -- 172.21.15.84:0/3929368629 learned_addr learned my addr 172.21.15.84:0/3929368629 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.061+0000 7fc6e1d9b700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0b1a80 0x7fc6dc1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e159a700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6dc0b81c0 0x7fc6dc153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e259c700 1 -- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0b1a80 msgr2=0x7fc6dc1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e259c700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0b1a80 0x7fc6dc1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e259c700 1 -- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6dc0b81c0 msgr2=0x7fc6dc153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e259c700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6dc0b81c0 0x7fc6dc153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e259c700 1 -- 172.21.15.84:0/3929368629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc009550 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e259c700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 0x7fc6dc0b7330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400efc0 tx=0x7fc6d400c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400cad0 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d400f040 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc0cb700 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400ce70 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.062+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc1542d0 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.063+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc04e1d0 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.064+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6d4014480 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.066+0000 7fc6d27fc700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6c006ace0 0x7fc6c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.066+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6d4089bd0 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.066+0000 7fc6e1d9b700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6c006ace0 0x7fc6c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.067+0000 7fc6e1d9b700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6c006ace0 0x7fc6c006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15cd20 tx=0x7fc6d8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.070+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6d4054be0 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.272+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6dc0b4f30 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.272+0000 7fc6d27fc700 1 -- 172.21.15.84:0/3929368629 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6d4058200 con 0x7fc6dc0ceda0 2023-07-06T21:39:42.276 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:39:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6c006ace0 msgr2=0x7fc6c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6c006ace0 0x7fc6c006d190 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15cd20 tx=0x7fc6d8009210 comp rx=0 tx=0).stop 2023-07-06T21:39:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 msgr2=0x7fc6dc0b7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 0x7fc6dc0b7330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc6d400efc0 tx=0x7fc6d400c0c0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 shutdown_connections 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6c006ace0 0x7fc6c006d190 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6dc0ceda0 0x7fc6dc0b7330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6dc0b1a80 0x7fc6dc1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 --2- 172.21.15.84:0/3929368629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6dc0b81c0 0x7fc6dc153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 >> 172.21.15.84:0/3929368629 conn(0x7fc6dc06b2b0 msgr2=0x7fc6dc0c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 shutdown_connections 2023-07-06T21:39:42.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.275+0000 7fc6e3fff700 1 -- 172.21.15.84:0/3929368629 wait complete. 2023-07-06T21:39:42.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:39:42.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.493+0000 7f97ff59e700 1 Processor -- start 2023-07-06T21:39:42.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.493+0000 7f97ff59e700 1 -- start start 2023-07-06T21:39:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.494+0000 7f97ff59e700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98000b8290 0x7f980006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.494+0000 7f97ff59e700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9800145410 0x7f9800145880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.494+0000 7f97ff59e700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f98000b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.494+0000 7f97ff59e700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980015c770 con 0x7f98000b8290 2023-07-06T21:39:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.494+0000 7f97ff59e700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9800146240 con 0x7f9800145410 2023-07-06T21:39:42.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.494+0000 7f97ff59e700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98000cf7a0 con 0x7f9800145dc0 2023-07-06T21:39:42.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98000b8290 0x7f980006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98000b8290 0x7f980006bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51328/0 (socket says 172.21.15.84:51328) 2023-07-06T21:39:42.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 -- 172.21.15.84:0/1318578044 learned_addr learned my addr 172.21.15.84:0/1318578044 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:42.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fed9d700 1 --2- 172.21.15.84:0/1318578044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f98000b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97f7fff700 1 --2- 172.21.15.84:0/1318578044 >> v2:172.21.15.107:3300/0 conn(0x7f9800145410 0x7f9800145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 -- 172.21.15.84:0/1318578044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 msgr2=0x7f98000b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 --2- 172.21.15.84:0/1318578044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f98000b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 -- 172.21.15.84:0/1318578044 >> v2:172.21.15.107:3300/0 conn(0x7f9800145410 msgr2=0x7f9800145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 --2- 172.21.15.84:0/1318578044 >> v2:172.21.15.107:3300/0 conn(0x7f9800145410 0x7f9800145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 -- 172.21.15.84:0/1318578044 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98000cd780 con 0x7f98000b8290 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97f7fff700 1 --2- 172.21.15.84:0/1318578044 >> v2:172.21.15.107:3300/0 conn(0x7f9800145410 0x7f9800145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.495+0000 7f97fe59c700 1 --2- 172.21.15.84:0/1318578044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f980006bc20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98000b54c0 tx=0x7f97f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb59647b664bb67 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97fdd9b700 1 -- 172.21.15.84:0/1318578044 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0005e30 con 0x7f98000b8290 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97fdd9b700 1 -- 172.21.15.84:0/1318578044 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0004030 con 0x7f98000b8290 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97fdd9b700 1 -- 172.21.15.84:0/1318578044 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00043e0 con 0x7f98000b8290 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 -- 172.21.15.84:0/1318578044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 msgr2=0x7f980006bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 --2- 172.21.15.84:0/1318578044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f980006bc20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f98000b54c0 tx=0x7f97f0009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 -- 172.21.15.84:0/1318578044 shutdown_connections 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 --2- 172.21.15.84:0/1318578044 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f98000b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 --2- 172.21.15.84:0/1318578044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f980006bc20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 --2- 172.21.15.84:0/1318578044 >> v2:172.21.15.107:3300/0 conn(0x7f9800145410 0x7f9800145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 -- 172.21.15.84:0/1318578044 >> 172.21.15.84:0/1318578044 conn(0x7f9800072220 msgr2=0x7f980006b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 -- 172.21.15.84:0/1318578044 shutdown_connections 2023-07-06T21:39:42.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.496+0000 7f97ff59e700 1 -- 172.21.15.84:0/1318578044 wait complete. 2023-07-06T21:39:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.498+0000 7f97ff59e700 1 Processor -- start 2023-07-06T21:39:42.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.498+0000 7f97ff59e700 1 -- start start 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97ff59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f98000cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97ff59e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800145410 0x7f9800158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97ff59e700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f9800158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97ff59e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98001594c0 con 0x7f98000b8290 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97ff59e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b6080 con 0x7f9800145410 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97ff59e700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98000b24c0 con 0x7f9800145dc0 2023-07-06T21:39:42.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800145410 0x7f9800158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97fe59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f98000cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800145410 0x7f9800158a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:59046/0 (socket says 172.21.15.84:59046) 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97f7fff700 1 -- 172.21.15.84:0/2571487534 learned_addr learned my addr 172.21.15.84:0/2571487534 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.499+0000 7f97fed9d700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f9800158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97fe59c700 1 -- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 msgr2=0x7f9800158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97fe59c700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f9800158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97fe59c700 1 -- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800145410 msgr2=0x7f9800158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97fe59c700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800145410 0x7f9800158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97fe59c700 1 -- 172.21.15.84:0/2571487534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0009500 con 0x7f98000b8290 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97fe59c700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f98000cee20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009fd0 tx=0x7f97f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0003c70 con 0x7f98000b8290 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0003df0 con 0x7f98000b8290 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98000b27a0 con 0x7f98000b8290 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.500+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0011660 con 0x7f98000b8290 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.501+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98000b5ca0 con 0x7f98000b8290 2023-07-06T21:39:42.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.502+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98000b2320 con 0x7f98000b8290 2023-07-06T21:39:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.502+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f97f00117c0 con 0x7f98000b8290 2023-07-06T21:39:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.503+0000 7f97f6ffd700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97e006ace0 0x7f97e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.503+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f97f008a580 con 0x7f98000b8290 2023-07-06T21:39:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.503+0000 7f97f7fff700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97e006ace0 0x7f97e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.504+0000 7f97f7fff700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97e006ace0 0x7f97e006d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98000c9370 tx=0x7f97e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.507+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98000b2320 con 0x7f98000b8290 2023-07-06T21:39:42.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.704+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98000b2320 con 0x7f98000b8290 2023-07-06T21:39:42.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.704+0000 7f97f6ffd700 1 -- 172.21.15.84:0/2571487534 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98000b2320 con 0x7f98000b8290 2023-07-06T21:39:42.709 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:39:42.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97e006ace0 msgr2=0x7f97e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97e006ace0 0x7f97e006d190 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f98000c9370 tx=0x7f97e8008040 comp rx=0 tx=0).stop 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 msgr2=0x7f98000cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f98000cee20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009fd0 tx=0x7f97f0004750 comp rx=0 tx=0).stop 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 shutdown_connections 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f97e006ace0 0x7f97e006d190 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9800145dc0 0x7f9800158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98000b8290 0x7f98000cee20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 --2- 172.21.15.84:0/2571487534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9800145410 0x7f9800158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 >> 172.21.15.84:0/2571487534 conn(0x7f9800072220 msgr2=0x7f980006b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:42.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 shutdown_connections 2023-07-06T21:39:42.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.707+0000 7f97ff59e700 1 -- 172.21.15.84:0/2571487534 wait complete. 2023-07-06T21:39:42.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:39:42.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.916+0000 7f45451ff700 1 Processor -- start 2023-07-06T21:39:42.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.916+0000 7f45451ff700 1 -- start start 2023-07-06T21:39:42.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.917+0000 7f45451ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f45400d3710 0x7f45400cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.917+0000 7f45451ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f45400cfa90 0x7f45400d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f45451ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400cd910 con 0x7f45400b83c0 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f45451ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400cff10 con 0x7f45400d3710 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f45451ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400d0a80 con 0x7f45400cfa90 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453ed9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f45400b83c0 0x7f45400cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46358/0 (socket says 172.21.15.84:46358) 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453ed9d700 1 -- 172.21.15.84:0/1622684222 learned_addr learned my addr 172.21.15.84:0/1622684222 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453e59c700 1 --2- 172.21.15.84:0/1622684222 >> v2:172.21.15.84:3300/0 conn(0x7f45400d3710 0x7f45400cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453f59e700 1 --2- 172.21.15.84:0/1622684222 >> v2:172.21.15.107:3300/0 conn(0x7f45400cfa90 0x7f45400d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453dd9b700 1 -- 172.21.15.84:0/1622684222 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387377503 0 0) 0x7f4530001110 con 0x7f45400b83c0 2023-07-06T21:39:42.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453e59c700 1 -- 172.21.15.84:0/1622684222 >> v2:172.21.15.107:3300/0 conn(0x7f45400cfa90 msgr2=0x7f45400d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:42.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453e59c700 1 --2- 172.21.15.84:0/1622684222 >> v2:172.21.15.107:3300/0 conn(0x7f45400cfa90 0x7f45400d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453e59c700 1 -- 172.21.15.84:0/1622684222 >> v1:172.21.15.84:6789/0 conn(0x7f45400b83c0 legacy=0x7f45400cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453e59c700 1 -- 172.21.15.84:0/1622684222 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400b8790 con 0x7f45400d3710 2023-07-06T21:39:42.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.918+0000 7f453e59c700 1 --2- 172.21.15.84:0/1622684222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400d3710 0x7f45400cf550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45400b1b00 tx=0x7f4534009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc324a62c2a2ac9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f453dd9b700 1 -- 172.21.15.84:0/1622684222 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534011830 con 0x7f45400d3710 2023-07-06T21:39:42.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f453dd9b700 1 -- 172.21.15.84:0/1622684222 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534011990 con 0x7f45400d3710 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f453dd9b700 1 -- 172.21.15.84:0/1622684222 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534011d20 con 0x7f45400d3710 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 -- 172.21.15.84:0/1622684222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400d3710 msgr2=0x7f45400cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 --2- 172.21.15.84:0/1622684222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400d3710 0x7f45400cf550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f45400b1b00 tx=0x7f4534009b20 comp rx=0 tx=0).stop 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 -- 172.21.15.84:0/1622684222 shutdown_connections 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 --2- 172.21.15.84:0/1622684222 >> v2:172.21.15.107:3300/0 conn(0x7f45400cfa90 0x7f45400d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 --2- 172.21.15.84:0/1622684222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400d3710 0x7f45400cf550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 -- 172.21.15.84:0/1622684222 >> 172.21.15.84:0/1622684222 conn(0x7f454006b350 msgr2=0x7f454006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:42.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 -- 172.21.15.84:0/1622684222 shutdown_connections 2023-07-06T21:39:42.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.919+0000 7f45451ff700 1 -- 172.21.15.84:0/1622684222 wait complete. 2023-07-06T21:39:42.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.921+0000 7f45451ff700 1 Processor -- start 2023-07-06T21:39:42.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.921+0000 7f45451ff700 1 -- start start 2023-07-06T21:39:42.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f45451ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45400b83c0 0x7f4540153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f45451ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 0x7f45400b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f45451ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45400d3710 0x7f45400b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f45451ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45401f3370 con 0x7f45400cfa90 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f45451ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454015e610 con 0x7f45400b83c0 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f45451ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45401f3090 con 0x7f45400d3710 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f453e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 0x7f45400b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f453e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 0x7f45400b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35950/0 (socket says 172.21.15.84:35950) 2023-07-06T21:39:42.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f453e59c700 1 -- 172.21.15.84:0/1732229716 learned_addr learned my addr 172.21.15.84:0/1732229716 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:42.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f453ed9d700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45400b83c0 0x7f4540153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.922+0000 7f453f59e700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45400d3710 0x7f45400b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453e59c700 1 -- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45400d3710 msgr2=0x7f45400b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453e59c700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45400d3710 0x7f45400b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453e59c700 1 -- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45400b83c0 msgr2=0x7f4540153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453e59c700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45400b83c0 0x7f4540153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453e59c700 1 -- 172.21.15.84:0/1732229716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340094a0 con 0x7f45400cfa90 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453f59e700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45400d3710 0x7f45400b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f453e59c700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 0x7f45400b69c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4534013000 tx=0x7f4534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45340036a0 con 0x7f45400cfa90 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453401f070 con 0x7f45400cfa90 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45400cb7d0 con 0x7f45400cfa90 2023-07-06T21:39:42.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534003a20 con 0x7f45400cfa90 2023-07-06T21:39:42.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.923+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45400cbd90 con 0x7f45400cfa90 2023-07-06T21:39:42.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.924+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400cb630 con 0x7f45400cfa90 2023-07-06T21:39:42.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.925+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4534003b80 con 0x7f45400cfa90 2023-07-06T21:39:42.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.926+0000 7f45277fe700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f452c06ace0 0x7f452c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:42.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.926+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4534021070 con 0x7f45400cfa90 2023-07-06T21:39:42.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.926+0000 7f453ed9d700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f452c06ace0 0x7f452c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:42.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.927+0000 7f453ed9d700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f452c06ace0 0x7f452c06d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4530000f60 tx=0x7f4530007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:42.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:42.930+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f45400cb630 con 0x7f45400cfa90 2023-07-06T21:39:43.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.171+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45400cb630 con 0x7f45400cfa90 2023-07-06T21:39:43.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.172+0000 7f45277fe700 1 -- 172.21.15.84:0/1732229716 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f45400cb630 con 0x7f45400cfa90 2023-07-06T21:39:43.175 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:43.175 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:39:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f452c06ace0 msgr2=0x7f452c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f452c06ace0 0x7f452c06d190 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4530000f60 tx=0x7f4530007e70 comp rx=0 tx=0).stop 2023-07-06T21:39:43.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 msgr2=0x7f45400b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 0x7f45400b69c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4534013000 tx=0x7f4534004750 comp rx=0 tx=0).stop 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 shutdown_connections 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f452c06ace0 0x7f452c06d190 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45400d3710 0x7f45400b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45400b83c0 0x7f4540153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 --2- 172.21.15.84:0/1732229716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45400cfa90 0x7f45400b69c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 >> 172.21.15.84:0/1732229716 conn(0x7f454006b350 msgr2=0x7f45400d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 shutdown_connections 2023-07-06T21:39:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.174+0000 7f45451ff700 1 -- 172.21.15.84:0/1732229716 wait complete. 2023-07-06T21:39:43.179 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:39:43.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:39:43.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.385+0000 7fa2cbfff700 1 Processor -- start 2023-07-06T21:39:43.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.385+0000 7fa2cbfff700 1 -- start start 2023-07-06T21:39:43.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2cbfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa2c40b83c0 0x7fa2c40cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2cbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2cbfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2c40d0a80 con 0x7fa2c40d3710 2023-07-06T21:39:43.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2cbfff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2c40b8790 con 0x7fa2c40b83c0 2023-07-06T21:39:43.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2cbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c40b4e70 con 0x7fa2c40cfc60 2023-07-06T21:39:43.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2c9d9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa2c40b83c0 0x7fa2c40cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2ca59c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2c9d9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa2c40b83c0 0x7fa2c40cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:56462/0 (socket says 172.21.15.84:56462) 2023-07-06T21:39:43.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.387+0000 7fa2c9d9b700 1 -- 172.21.15.84:0/1340137887 learned_addr learned my addr 172.21.15.84:0/1340137887 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:43.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c9d9b700 1 -- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 msgr2=0x7fa2c40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:43.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c9d9b700 1 --2- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c9d9b700 1 -- 172.21.15.84:0/1340137887 >> v1:172.21.15.107:6789/0 conn(0x7fa2c40d3710 legacy=0x7fa2c40cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c9d9b700 1 -- 172.21.15.84:0/1340137887 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c40b5150 con 0x7fa2c40b83c0 2023-07-06T21:39:43.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2ca59c700 1 --2- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:43.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c8d99700 1 -- 172.21.15.84:0/1340137887 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2360192566 0 0) 0x7fa2c40d0a80 con 0x7fa2c40d3710 2023-07-06T21:39:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c9d9b700 1 --2- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c40cd200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00098f0 tx=0x7fa2c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a9a5f7caf8e9fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c8d99700 1 -- 172.21.15.84:0/1340137887 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0011830 con 0x7fa2c40b83c0 2023-07-06T21:39:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c8d99700 1 -- 172.21.15.84:0/1340137887 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c0011990 con 0x7fa2c40b83c0 2023-07-06T21:39:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.388+0000 7fa2c8d99700 1 -- 172.21.15.84:0/1340137887 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0011d20 con 0x7fa2c40b83c0 2023-07-06T21:39:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 msgr2=0x7fa2c40cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c40cd200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00098f0 tx=0x7fa2c0009c00 comp rx=0 tx=0).stop 2023-07-06T21:39:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1340137887 shutdown_connections 2023-07-06T21:39:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1340137887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c40cd200 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1340137887 >> 172.21.15.84:0/1340137887 conn(0x7fa2c406b350 msgr2=0x7fa2c406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1340137887 shutdown_connections 2023-07-06T21:39:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.389+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1340137887 wait complete. 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.391+0000 7fa2cbfff700 1 Processor -- start 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.391+0000 7fa2cbfff700 1 -- start start 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2cbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2cbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2cbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 0x7fa2c41f13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2cbfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41f52c0 con 0x7fa2c40d3710 2023-07-06T21:39:43.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2cbfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41f55a0 con 0x7fa2c40b83c0 2023-07-06T21:39:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2cbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c41f5880 con 0x7fa2c40cfc60 2023-07-06T21:39:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 0x7fa2c41f13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 0x7fa2c41f13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35966/0 (socket says 172.21.15.84:35966) 2023-07-06T21:39:43.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 -- 172.21.15.84:0/1247145733 learned_addr learned my addr 172.21.15.84:0/1247145733 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2c9d9b700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2c959a700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 -- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 msgr2=0x7fa2c40cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 -- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 msgr2=0x7fa2c415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c415f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2ca59c700 1 -- 172.21.15.84:0/1247145733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc009a70 con 0x7fa2c40d3710 2023-07-06T21:39:43.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.392+0000 7fa2c959a700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.393+0000 7fa2ca59c700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 0x7fa2c41f13d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00efb0 tx=0x7fa2bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.393+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc009cb0 con 0x7fa2c40d3710 2023-07-06T21:39:43.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.393+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2bc00f040 con 0x7fa2c40d3710 2023-07-06T21:39:43.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.393+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc014480 con 0x7fa2c40d3710 2023-07-06T21:39:43.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.393+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c0009550 con 0x7fa2c40d3710 2023-07-06T21:39:43.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.393+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c40c9e50 con 0x7fa2c40d3710 2023-07-06T21:39:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.394+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c41f6080 con 0x7fa2c40d3710 2023-07-06T21:39:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.395+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa2bc0145e0 con 0x7fa2c40d3710 2023-07-06T21:39:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.396+0000 7fa2ba7fc700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2a806ace0 0x7fa2a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.396+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa2bc08a170 con 0x7fa2c40d3710 2023-07-06T21:39:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.396+0000 7fa2c9d9b700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2a806ace0 0x7fa2a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.397+0000 7fa2c9d9b700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2a806ace0 0x7fa2a806d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0014040 tx=0x7fa2c0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:43.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.400+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa2c41f6080 con 0x7fa2c40d3710 2023-07-06T21:39:43.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.598+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2c41f6080 con 0x7fa2c40d3710 2023-07-06T21:39:43.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.599+0000 7fa2ba7fc700 1 -- 172.21.15.84:0/1247145733 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7fa2c41f6080 con 0x7fa2c40d3710 2023-07-06T21:39:43.603 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:43.603 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:39:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.601+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2a806ace0 msgr2=0x7fa2a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.601+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2a806ace0 0x7fa2a806d190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0014040 tx=0x7fa2c0008030 comp rx=0 tx=0).stop 2023-07-06T21:39:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.601+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 msgr2=0x7fa2c41f13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.601+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 0x7fa2c41f13d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00efb0 tx=0x7fa2bc00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 shutdown_connections 2023-07-06T21:39:43.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2a806ace0 0x7fa2a806d190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa2c40d3710 0x7fa2c41f13d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2c40b83c0 0x7fa2c415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 --2- 172.21.15.84:0/1247145733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa2c40cfc60 0x7fa2c40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 >> 172.21.15.84:0/1247145733 conn(0x7fa2c406b350 msgr2=0x7fa2c40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 shutdown_connections 2023-07-06T21:39:43.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.602+0000 7fa2cbfff700 1 -- 172.21.15.84:0/1247145733 wait complete. 2023-07-06T21:39:43.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:39:43.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.815+0000 7f4d22bff700 1 Processor -- start 2023-07-06T21:39:43.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.815+0000 7f4d22bff700 1 -- start start 2023-07-06T21:39:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.816+0000 7f4d22bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4d1c0b83c0 0x7f4d1c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.816+0000 7f4d22bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d1c0d3710 0x7f4d1c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.816+0000 7f4d22bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.816+0000 7f4d22bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cff10 con 0x7f4d1c0d3710 2023-07-06T21:39:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.816+0000 7f4d22bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0d0a80 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.816+0000 7f4d22bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0b8790 con 0x7f4d1c0cfa90 2023-07-06T21:39:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d1c0d3710 0x7f4d1c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4d1c0d3710 0x7f4d1c0cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35976/0 (socket says 172.21.15.84:35976) 2023-07-06T21:39:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 -- 172.21.15.84:0/2505950247 learned_addr learned my addr 172.21.15.84:0/2505950247 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d2099b700 1 --2- 172.21.15.84:0/2505950247 >> v2:172.21.15.107:3300/0 conn(0x7f4d1c0b83c0 0x7f4d1c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d2119c700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 -- 172.21.15.84:0/2505950247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 msgr2=0x7f4d1c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 -- 172.21.15.84:0/2505950247 >> v2:172.21.15.107:3300/0 conn(0x7f4d1c0b83c0 msgr2=0x7f4d1c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 --2- 172.21.15.84:0/2505950247 >> v2:172.21.15.107:3300/0 conn(0x7f4d1c0b83c0 0x7f4d1c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 -- 172.21.15.84:0/2505950247 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0b4e70 con 0x7f4d1c0d3710 2023-07-06T21:39:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d2099b700 1 --2- 172.21.15.84:0/2505950247 >> v2:172.21.15.107:3300/0 conn(0x7f4d1c0b83c0 0x7f4d1c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:43.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d2119c700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1bfff700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c0cf550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0cf520 tx=0x7f4d08009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc355f20f71ed4e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1b7fe700 1 -- 172.21.15.84:0/2505950247 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800b870 con 0x7f4d1c0d3710 2023-07-06T21:39:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.817+0000 7f4d1b7fe700 1 -- 172.21.15.84:0/2505950247 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800b9d0 con 0x7f4d1c0d3710 2023-07-06T21:39:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d1b7fe700 1 -- 172.21.15.84:0/2505950247 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800bd60 con 0x7f4d1c0d3710 2023-07-06T21:39:43.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 -- 172.21.15.84:0/2505950247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0d3710 msgr2=0x7f4d1c0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c0cf550 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0cf520 tx=0x7f4d08009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 -- 172.21.15.84:0/2505950247 shutdown_connections 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 --2- 172.21.15.84:0/2505950247 >> v2:172.21.15.107:3300/0 conn(0x7f4d1c0b83c0 0x7f4d1c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 --2- 172.21.15.84:0/2505950247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c0cf550 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 -- 172.21.15.84:0/2505950247 >> 172.21.15.84:0/2505950247 conn(0x7f4d1c06b350 msgr2=0x7f4d1c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 -- 172.21.15.84:0/2505950247 shutdown_connections 2023-07-06T21:39:43.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.818+0000 7f4d22bff700 1 -- 172.21.15.84:0/2505950247 wait complete. 2023-07-06T21:39:43.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.821+0000 7f4d22bff700 1 Processor -- start 2023-07-06T21:39:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.821+0000 7f4d22bff700 1 -- start start 2023-07-06T21:39:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d22bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 0x7f4d1c1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d22bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c0c7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d22bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c1eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d22bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c8ee0 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d22bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c82d0 con 0x7f4d1c0d3710 2023-07-06T21:39:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d22bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c85b0 con 0x7f4d1c0cfa90 2023-07-06T21:39:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d2099b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 0x7f4d1c1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d2099b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 0x7f4d1c1f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35988/0 (socket says 172.21.15.84:35988) 2023-07-06T21:39:43.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d2099b700 1 -- 172.21.15.84:0/1073571847 learned_addr learned my addr 172.21.15.84:0/1073571847 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d1bfff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c0c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.822+0000 7f4d2119c700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c1eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2099b700 1 -- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 msgr2=0x7f4d1c0c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2099b700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c0c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2099b700 1 -- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c0d3710 msgr2=0x7f4d1c1eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2099b700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c1eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2099b700 1 -- 172.21.15.84:0/1073571847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08009550 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2119c700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c1eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d2099b700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 0x7f4d1c1f5800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c7f0 tx=0x7f4d1000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10011800 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10011960 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c0c8890 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10011cf0 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.823+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c0c9370 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.824+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c0b4930 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.825+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4d10018360 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.826+0000 7f4d197fa700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4d0006ace0 0x7f4d0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:43.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.826+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4d10089540 con 0x7f4d1c0b83c0 2023-07-06T21:39:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.827+0000 7f4d1bfff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4d0006ace0 0x7f4d0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.827+0000 7f4d1bfff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4d0006ace0 0x7f4d0006d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0d3b20 tx=0x7f4d08008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:43.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:43.831+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4d10054560 con 0x7f4d1c0b83c0 2023-07-06T21:39:44.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.072+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d1c0d1500 con 0x7f4d1c0b83c0 2023-07-06T21:39:44.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.072+0000 7f4d197fa700 1 -- 172.21.15.84:0/1073571847 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f4d10057b80 con 0x7f4d1c0b83c0 2023-07-06T21:39:44.077 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:39:44.077 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:39:44.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4d0006ace0 msgr2=0x7f4d0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4d0006ace0 0x7f4d0006d190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0d3b20 tx=0x7f4d08008030 comp rx=0 tx=0).stop 2023-07-06T21:39:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 msgr2=0x7f4d1c1f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 0x7f4d1c1f5800 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c7f0 tx=0x7f4d1000cb00 comp rx=0 tx=0).stop 2023-07-06T21:39:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 shutdown_connections 2023-07-06T21:39:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4d0006ace0 0x7f4d0006d190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d1c0d3710 0x7f4d1c1eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4d1c0b83c0 0x7f4d1c1f5800 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 --2- 172.21.15.84:0/1073571847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1c0cfa90 0x7f4d1c0c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 >> 172.21.15.84:0/1073571847 conn(0x7f4d1c06b350 msgr2=0x7f4d1c159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 shutdown_connections 2023-07-06T21:39:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.075+0000 7f4d22bff700 1 -- 172.21.15.84:0/1073571847 wait complete. 2023-07-06T21:39:44.081 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:39:44.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T21:39:26.427365+0000', 'modified': '2023-07-06T21:39:29.915595+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6835/2897792334', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 2897792334}, {'type': 'v1', 'addr': '172.21.15.107:6835', 'nonce': 2897792334}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T21:39:44.499 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T21:39:44.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T21:39:44.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.691+0000 7f77347ff700 1 Processor -- start 2023-07-06T21:39:44.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.691+0000 7f77347ff700 1 -- start start 2023-07-06T21:39:44.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f77347ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f772c0b83c0 0x7f772c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f77347ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f77347ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f772c0cfa90 0x7f772c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f77347ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0cfee0 con 0x7f772c0cfa90 2023-07-06T21:39:44.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f77347ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f772c0d0a80 con 0x7f772c0b83c0 2023-07-06T21:39:44.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f77347ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c0b8790 con 0x7f772c0d3710 2023-07-06T21:39:44.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f7732d9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f772c0cfa90 0x7f772c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f7732d9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f772c0cfa90 0x7f772c15feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36000/0 (socket says 172.21.15.84:36000) 2023-07-06T21:39:44.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f7732d9c700 1 -- 172.21.15.84:0/88408598 learned_addr learned my addr 172.21.15.84:0/88408598 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:44.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f7731d9a700 1 --2- 172.21.15.84:0/88408598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.692+0000 7f773259b700 1 --2- 172.21.15.84:0/88408598 >> v2:172.21.15.107:3300/0 conn(0x7f772c0b83c0 0x7f772c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7732d9c700 1 -- 172.21.15.84:0/88408598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 msgr2=0x7f772c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7732d9c700 1 --2- 172.21.15.84:0/88408598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7732d9c700 1 -- 172.21.15.84:0/88408598 >> v2:172.21.15.107:3300/0 conn(0x7f772c0b83c0 msgr2=0x7f772c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7732d9c700 1 --2- 172.21.15.84:0/88408598 >> v2:172.21.15.107:3300/0 conn(0x7f772c0b83c0 0x7f772c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7732d9c700 1 -- 172.21.15.84:0/88408598 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c0b4e70 con 0x7f772c0cfa90 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7732d9c700 1 --2- 172.21.15.84:0/88408598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c15feb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f772c0cf520 tx=0x7f772400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23ee0f2f36d9a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7731599700 1 -- 172.21.15.84:0/88408598 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400e070 con 0x7f772c0cfa90 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7731599700 1 -- 172.21.15.84:0/88408598 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772400b910 con 0x7f772c0cfa90 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.693+0000 7f7731599700 1 -- 172.21.15.84:0/88408598 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400bcc0 con 0x7f772c0cfa90 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 -- 172.21.15.84:0/88408598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 msgr2=0x7f772c15feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 --2- 172.21.15.84:0/88408598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c15feb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f772c0cf520 tx=0x7f772400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 -- 172.21.15.84:0/88408598 shutdown_connections 2023-07-06T21:39:44.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 --2- 172.21.15.84:0/88408598 >> v2:172.21.15.107:3300/0 conn(0x7f772c0b83c0 0x7f772c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 --2- 172.21.15.84:0/88408598 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c15feb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 --2- 172.21.15.84:0/88408598 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 -- 172.21.15.84:0/88408598 >> 172.21.15.84:0/88408598 conn(0x7f772c06b350 msgr2=0x7f772c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 -- 172.21.15.84:0/88408598 shutdown_connections 2023-07-06T21:39:44.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.694+0000 7f77347ff700 1 -- 172.21.15.84:0/88408598 wait complete. 2023-07-06T21:39:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.696+0000 7f77347ff700 1 Processor -- start 2023-07-06T21:39:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.696+0000 7f77347ff700 1 -- start start 2023-07-06T21:39:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f77347ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f772c0b83c0 0x7f772c14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f77347ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c0c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f77347ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f77347ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c15bc40 con 0x7f772c0cfa90 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f77347ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c15be90 con 0x7f772c0b83c0 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f77347ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c15c170 con 0x7f772c0d3710 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c0c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c0c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36014/0 (socket says 172.21.15.84:36014) 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f7731d9a700 1 -- 172.21.15.84:0/2907117304 learned_addr learned my addr 172.21.15.84:0/2907117304 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f773259b700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f772c0b83c0 0x7f772c14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.697+0000 7f7732d9c700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7731d9a700 1 -- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 msgr2=0x7f772c0b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7731d9a700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7731d9a700 1 -- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f772c0b83c0 msgr2=0x7f772c14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7731d9a700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f772c0b83c0 0x7f772c14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7731d9a700 1 -- 172.21.15.84:0/2907117304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009b30 con 0x7f772c0cfa90 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f773259b700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f772c0b83c0 0x7f772c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7731d9a700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c0c8c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e840 tx=0x7f771c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7722ffd700 1 -- 172.21.15.84:0/2907117304 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c009590 con 0x7f772c0cfa90 2023-07-06T21:39:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7722ffd700 1 -- 172.21.15.84:0/2907117304 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c013680 con 0x7f772c0cfa90 2023-07-06T21:39:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f7722ffd700 1 -- 172.21.15.84:0/2907117304 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c013a30 con 0x7f772c0cfa90 2023-07-06T21:39:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c15c450 con 0x7f772c0cfa90 2023-07-06T21:39:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.698+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c0c9150 con 0x7f772c0cfa90 2023-07-06T21:39:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.700+0000 7f7722ffd700 1 -- 172.21.15.84:0/2907117304 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f771c013b90 con 0x7f772c0cfa90 2023-07-06T21:39:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.702+0000 7f7722ffd700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f771006ace0 0x7f771006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.702+0000 7f7722ffd700 1 -- 172.21.15.84:0/2907117304 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f771c089230 con 0x7f772c0cfa90 2023-07-06T21:39:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.702+0000 7f773259b700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f771006ace0 0x7f771006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.702+0000 7f773259b700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f771006ace0 0x7f771006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7728009b90 tx=0x7f7728008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.703+0000 7f77480d1b80 1 Processor -- start 2023-07-06T21:39:44.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.703+0000 7f77480d1b80 1 -- start start 2023-07-06T21:39:44.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e60320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e86820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e8a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f77480d1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65d64c60 con 0x562d65e5ff50 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65db79f0 con 0x562d65e87d80 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d65db0e80 con 0x562d65e5d740 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.704+0000 7f773259b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e60320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e60320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36024/0 (socket says 172.21.15.84:36024) 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 -- 172.21.15.84:0/4141400560 learned_addr learned my addr 172.21.15.84:0/4141400560 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f7732d9c700 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e8a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f7731d9a700 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e86820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 -- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 msgr2=0x562d65e8a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e8a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 -- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 msgr2=0x562d65e86820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e86820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 -- 172.21.15.84:0/4141400560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d65c325e0 con 0x562d65e5ff50 2023-07-06T21:39:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f773259b700 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e60320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7728004490 tx=0x7f7728014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18010793f72d470e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.705+0000 7f770ffff700 1 -- 172.21.15.84:0/4141400560 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280044c0 con 0x562d65e5ff50 2023-07-06T21:39:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f770ffff700 1 -- 172.21.15.84:0/4141400560 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77280036a0 con 0x562d65e5ff50 2023-07-06T21:39:44.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f770ffff700 1 -- 172.21.15.84:0/4141400560 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728003a10 con 0x562d65e5ff50 2023-07-06T21:39:44.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 -- 172.21.15.84:0/4141400560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 msgr2=0x562d65e60320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e60320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7728004490 tx=0x7f7728014000 comp rx=0 tx=0).stop 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 -- 172.21.15.84:0/4141400560 shutdown_connections 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e8a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e60320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 --2- 172.21.15.84:0/4141400560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e86820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 -- 172.21.15.84:0/4141400560 >> 172.21.15.84:0/4141400560 conn(0x562d65e54fc0 msgr2=0x562d65ea51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 -- 172.21.15.84:0/4141400560 shutdown_connections 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.706+0000 7f77480d1b80 1 -- 172.21.15.84:0/4141400560 wait complete. 2023-07-06T21:39:44.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.707+0000 7f77480d1b80 1 -- start start 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.707+0000 7f77480d1b80 1 Processor -- start 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e5d740 0x562d65eb4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e9d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e87d80 0x562d65e9dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65d50a70 con 0x562d65e5ff50 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65d50770 con 0x562d65e5d740 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d65c348d0 con 0x562d65e87d80 2023-07-06T21:39:44.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f77480d1b80 10 client.0 _ll_register_callbacks cb 0x562d65d56210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e9d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e9d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36038/0 (socket says 172.21.15.84:36038) 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.708+0000 7f7731d9a700 1 -- 172.21.15.84:0/452421276 learned_addr learned my addr 172.21.15.84:0/452421276 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f773259b700 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e5d740 0x562d65eb4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f7731d9a700 1 -- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e87d80 msgr2=0x562d65e9dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f7731d9a700 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e87d80 0x562d65e9dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f7731d9a700 1 -- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e5d740 msgr2=0x562d65eb4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f7731d9a700 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e5d740 0x562d65eb4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f7731d9a700 1 -- 172.21.15.84:0/452421276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7728004170 con 0x562d65e5ff50 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f7731d9a700 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e9d980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f771c00ef50 tx=0x7f771c05c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2952bd12c8abcc88 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f770d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f770d7fa700 1 -- 172.21.15.84:0/452421276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c056490 con 0x562d65e5ff50 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f770d7fa700 1 -- 172.21.15.84:0/452421276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f771c013e40 con 0x562d65e5ff50 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d65e52780 con 0x562d65e5ff50 2023-07-06T21:39:44.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f770d7fa700 1 -- 172.21.15.84:0/452421276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f771c087250 con 0x562d65e5ff50 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f77480d1b80 10 client.4650 fetch_fsmap learned FSMap version 8 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.709+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x562d65bded90 con 0x562d65e5ff50 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f770d7fa700 1 -- 172.21.15.84:0/452421276 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562d65e52780 con 0x562d65e5ff50 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f770d7fa700 1 -- 172.21.15.84:0/452421276 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f771c053e20 con 0x562d65e5ff50 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f77480d1b80 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f77480d1b80 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f77480d1b80 20 client.4650 populate_metadata read hostname 'smithi084' 2023-07-06T21:39:44.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f77480d1b80 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65eb8270 0x562d65ec1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f77480d1b80 4 client.4650 mds_command: new command op to 4300 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T21:39:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 --> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d65da9e60 con 0x562d65eb8270 2023-07-06T21:39:44.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.710+0000 7f773259b700 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65eb8270 0x562d65ec1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.711+0000 7f773259b700 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65eb8270 0x562d65ec1b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f275fbda7604888 server_cookie=a342bd4658283ce3 in_seq=0 out_seq=0 2023-07-06T21:39:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.711+0000 7f770d7fa700 10 client.4650 ms_handle_connect on v2:172.21.15.107:6834/2897792334 2023-07-06T21:39:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f770d7fa700 1 -- 172.21.15.84:0/452421276 <== mds.0 v2:172.21.15.107:6834/2897792334 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d65da9e60 con 0x562d65eb8270 2023-07-06T21:39:44.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f770d7fa700 10 client.4650 handle_command_reply: tid=0 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 client.4650 shutdown 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 msgr2=0x562d65e9d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e9d980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f771c00ef50 tx=0x7f771c05c170 comp rx=0 tx=0).stop 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 shutdown_connections 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e87d80 0x562d65e9dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e5d740 0x562d65eb4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65eb8270 0x562d65ec1b60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 --2- 172.21.15.84:0/452421276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e9d980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f770d7fa700 0 client.4650 ms_handle_reset on v2:172.21.15.107:6834/2897792334 2023-07-06T21:39:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 >> 172.21.15.84:0/452421276 conn(0x562d65e54fc0 msgr2=0x562d65e86ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 shutdown_connections 2023-07-06T21:39:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 1 -- 172.21.15.84:0/452421276 wait complete. 2023-07-06T21:39:44.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.712+0000 7f77480d1b80 20 client.4650 trim_cache size 0 max 16384 2023-07-06T21:39:44.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.730+0000 7f77480d1b80 1 Processor -- start 2023-07-06T21:39:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.730+0000 7f77480d1b80 1 -- start start 2023-07-06T21:39:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e9d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65ec5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e5a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f77480d1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65e5d050 con 0x562d65e5ff50 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65ea35f0 con 0x562d65e87d80 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d65e9da30 con 0x562d65e5d740 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.731+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65ec5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65ec5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36042/0 (socket says 172.21.15.84:36042) 2023-07-06T21:39:44.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 -- 172.21.15.84:0/2618684851 learned_addr learned my addr 172.21.15.84:0/2618684851 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:44.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 -- 172.21.15.84:0/2618684851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 msgr2=0x562d65e9d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e9d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 -- 172.21.15.84:0/2618684851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 msgr2=0x562d65e5a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e5a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 -- 172.21.15.84:0/2618684851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d65bc7c10 con 0x562d65e5ff50 2023-07-06T21:39:44.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7731d9a700 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65ec5b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f771c054530 tx=0x7f771c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5727a217f96882ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7720ff9700 1 -- 172.21.15.84:0/2618684851 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c087250 con 0x562d65e5ff50 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7720ff9700 1 -- 172.21.15.84:0/2618684851 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771c056490 con 0x562d65e5ff50 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.732+0000 7f7720ff9700 1 -- 172.21.15.84:0/2618684851 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c056f90 con 0x562d65e5ff50 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 -- 172.21.15.84:0/2618684851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 msgr2=0x562d65ec5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65ec5b80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f771c054530 tx=0x7f771c01d000 comp rx=0 tx=0).stop 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 -- 172.21.15.84:0/2618684851 shutdown_connections 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65e5a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e9d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 --2- 172.21.15.84:0/2618684851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65ec5b80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 -- 172.21.15.84:0/2618684851 >> 172.21.15.84:0/2618684851 conn(0x562d65e54fc0 msgr2=0x562d65e9de30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 -- 172.21.15.84:0/2618684851 shutdown_connections 2023-07-06T21:39:44.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 -- 172.21.15.84:0/2618684851 wait complete. 2023-07-06T21:39:44.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.733+0000 7f77480d1b80 1 -- start start 2023-07-06T21:39:44.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.734+0000 7f77480d1b80 1 Processor -- start 2023-07-06T21:39:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e58f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65f0be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65ec5ec0 con 0x562d65e5ff50 2023-07-06T21:39:44.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d65ec5c10 con 0x562d65e87d80 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d65e81c90 con 0x562d65e5d740 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f77480d1b80 10 client.0 _ll_register_callbacks cb 0x562d65d56210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36052/0 (socket says 172.21.15.84:36052) 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 -- 172.21.15.84:0/702573182 learned_addr learned my addr 172.21.15.84:0/702573182 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f773259b700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e58f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7732d9c700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65f0be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 -- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 msgr2=0x562d65e58f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e58f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 -- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 msgr2=0x562d65f0be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65f0be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 -- 172.21.15.84:0/702573182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d65bded90 con 0x562d65e5ff50 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7732d9c700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65f0be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f7731d9a700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e594d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f771c00ef50 tx=0x7f771c05b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56725468fe3f809f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:44.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f770d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.735+0000 7f770d7fa700 1 -- 172.21.15.84:0/702573182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771c056f90 con 0x562d65e5ff50 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f770d7fa700 1 -- 172.21.15.84:0/702573182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f771c013e40 con 0x562d65e5ff50 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77280036a0 con 0x562d65e5ff50 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f770d7fa700 1 -- 172.21.15.84:0/702573182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f771c0570f0 con 0x562d65e5ff50 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 10 client.4656 fetch_fsmap learned FSMap version 8 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x562d65c05200 con 0x562d65e5ff50 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f770d7fa700 1 -- 172.21.15.84:0/702573182 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77280036a0 con 0x562d65e5ff50 2023-07-06T21:39:44.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f770d7fa700 1 -- 172.21.15.84:0/702573182 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f771c087a20 con 0x562d65e5ff50 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 20 client.4656 populate_metadata read hostname 'smithi084' 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65e82530 0x562d65f0c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 4 client.4656 mds_command: new command op to 4300 tid=0[{"prefix": "status"}] 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 --> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d65d6c000 con 0x562d65e82530 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.736+0000 7f773259b700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65e82530 0x562d65f0c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:44.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f773259b700 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65e82530 0x562d65f0c390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b57f50d19d2b586b server_cookie=6027f930401614f in_seq=0 out_seq=0 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f770d7fa700 10 client.4656 ms_handle_connect on v2:172.21.15.107:6834/2897792334 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f770d7fa700 1 -- 172.21.15.84:0/702573182 <== mds.0 v2:172.21.15.107:6834/2897792334 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x562d65d6c000 con 0x562d65e82530 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f770d7fa700 10 client.4656 handle_command_reply: tid=0 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 client.4656 shutdown 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 msgr2=0x562d65e594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e594d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f771c00ef50 tx=0x7f771c05b040 comp rx=0 tx=0).stop 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 shutdown_connections 2023-07-06T21:39:44.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d65e87d80 0x562d65f0be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d65e5d740 0x562d65e58f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.107:6834/2897792334,v1:172.21.15.107:6835/2897792334] conn(0x562d65e82530 0x562d65f0c390 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 --2- 172.21.15.84:0/702573182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562d65e5ff50 0x562d65e594d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f770d7fa700 0 client.4656 ms_handle_reset on v2:172.21.15.107:6834/2897792334 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.737+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 >> 172.21.15.84:0/702573182 conn(0x562d65e54fc0 msgr2=0x562d65eb38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 shutdown_connections 2023-07-06T21:39:44.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77480d1b80 1 -- 172.21.15.84:0/702573182 wait complete. 2023-07-06T21:39:44.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77480d1b80 20 client.4656 trim_cache size 0 max 16384 2023-07-06T21:39:44.749 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:39:44.749 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c2762452-5842-4ccc-b0b9-8dbf6ac718c9", 2023-07-06T21:39:44.749 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-07-06T21:39:44.749 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4300, 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 18.307991327, 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 8, 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 23, 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 23, 2023-07-06T21:39:44.750 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 24.034875052 2023-07-06T21:39:44.751 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:39:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f771006ace0 msgr2=0x7f771006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f771006ace0 0x7f771006d190 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7728009b90 tx=0x7f7728008040 comp rx=0 tx=0).stop 2023-07-06T21:39:44.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 msgr2=0x7f772c0c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c0c8c10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f771c00e840 tx=0x7f771c00eb50 comp rx=0 tx=0).stop 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 shutdown_connections 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f771006ace0 0x7f771006d190 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f772c0d3710 0x7f772c0b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f772c0b83c0 0x7f772c14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 --2- 172.21.15.84:0/2907117304 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f772c0cfa90 0x7f772c0c8c10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:44.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 >> 172.21.15.84:0/2907117304 conn(0x7f772c06b350 msgr2=0x7f772c0d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:44.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 shutdown_connections 2023-07-06T21:39:44.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:44.738+0000 7f77347ff700 1 -- 172.21.15.84:0/2907117304 wait complete. 2023-07-06T21:39:44.759 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-07-06T21:39:44.783 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-07-06T21:39:44.783 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-07-06T21:39:44.783 INFO:tasks.ceph_fuse:client.0 config is {} 2023-07-06T21:39:44.784 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T21:39:44.784 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:44.784 INFO:tasks.ceph_fuse:client.2 config is {} 2023-07-06T21:39:44.785 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T21:39:44.785 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:44.785 INFO:tasks.ceph_fuse:client.1 config is {} 2023-07-06T21:39:44.785 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T21:39:44.785 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T21:39:44.786 INFO:tasks.ceph_fuse:client.3 config is {} 2023-07-06T21:39:44.786 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-06T21:39:44.786 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T21:39:44.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:44.787 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:39:44.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:44.830 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-07-06T21:39:44.940 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2023-07-06T21:39:44.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:39:44.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:44.944 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-06T21:39:44.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:44.994 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-06T21:39:45.067 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2023-07-06T21:39:45.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:39:45.070 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-07-06T21:39:45.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:39:45.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.262+0000 7fc6ce5ff700 1 Processor -- start 2023-07-06T21:39:45.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.262+0000 7fc6ce5ff700 1 -- start start 2023-07-06T21:39:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.263+0000 7fc6ce5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc6c80d3710 0x7fc6c80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.263+0000 7fc6ce5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6c80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.263+0000 7fc6ce5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c80cd910 con 0x7fc6c80b83c0 2023-07-06T21:39:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.263+0000 7fc6ce5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c80cff10 con 0x7fc6c80d3710 2023-07-06T21:39:45.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.263+0000 7fc6ce5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80d0a80 con 0x7fc6c80cfa90 2023-07-06T21:39:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc6c80b83c0 0x7fc6c80cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46360/0 (socket says 172.21.15.84:46360) 2023-07-06T21:39:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c7fff700 1 -- 172.21.15.84:0/513108722 learned_addr learned my addr 172.21.15.84:0/513108722 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:45.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6ccb9c700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6c80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114880394 0 0) 0x7fc6b8001110 con 0x7fc6c80b83c0 2023-07-06T21:39:45.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/513108722 >> v2:172.21.15.107:3300/0 conn(0x7fc6c80d3710 0x7fc6c80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6b00035b0 con 0x7fc6c80b83c0 2023-07-06T21:39:45.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339250905 0 0) 0x7fc6b00035b0 con 0x7fc6c80b83c0 2023-07-06T21:39:45.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 msgr2=0x7fc6c80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:45.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6c80d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 >> v2:172.21.15.107:3300/0 conn(0x7fc6c80d3710 msgr2=0x7fc6c80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:45.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 --2- 172.21.15.84:0/513108722 >> v2:172.21.15.107:3300/0 conn(0x7fc6c80d3710 0x7fc6c80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c80b8790 con 0x7fc6c80b83c0 2023-07-06T21:39:45.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fc6b8001800 con 0x7fc6c80b83c0 2023-07-06T21:39:45.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 >> v1:172.21.15.84:6789/0 conn(0x7fc6c80b83c0 legacy=0x7fc6c80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6b0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b0003f80 0x7fc6b0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80cff10 con 0x7fc6b0004d90 2023-07-06T21:39:45.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80d0a80 con 0x7fc6b0003f80 2023-07-06T21:39:45.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.264+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b000ba40 con 0x7fc6c80cfa90 2023-07-06T21:39:45.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b0003f80 0x7fc6b0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6ccb9c700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c7fff700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6b0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 -- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 msgr2=0x7fc6b0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:45.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6b0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 -- 172.21.15.84:0/513108722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0004d90 msgr2=0x7fc6b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:45.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 -- 172.21.15.84:0/513108722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b00035b0 con 0x7fc6b0003f80 2023-07-06T21:39:45.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c7fff700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cfa90 0x7fc6b0003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:39:45.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6ccb9c700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0004d90 0x7fc6b0005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:45.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.265+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b0003f80 0x7fc6b0004850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009b30 tx=0x7fc6b0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b6391a99e671ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:45.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00e6c0 con 0x7fc6b0003f80 2023-07-06T21:39:45.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00e820 con 0x7fc6b0003f80 2023-07-06T21:39:45.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6c6ffd700 1 -- 172.21.15.84:0/513108722 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00eb90 con 0x7fc6b0003f80 2023-07-06T21:39:45.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b0003f80 msgr2=0x7fc6b0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:45.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/513108722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b0003f80 0x7fc6b0004850 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009b30 tx=0x7fc6b0005880 comp rx=0 tx=0).stop 2023-07-06T21:39:45.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6ccb9c700 1 -- 172.21.15.84:0/513108722 reap_dead start 2023-07-06T21:39:45.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/513108722 shutdown_connections 2023-07-06T21:39:45.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/513108722 >> 172.21.15.84:0/513108722 conn(0x7fc6c806b350 msgr2=0x7fc6c806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:45.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.266+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/513108722 shutdown_connections 2023-07-06T21:39:45.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.267+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/513108722 wait complete. 2023-07-06T21:39:45.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.268+0000 7fc6ce5ff700 1 Processor -- start 2023-07-06T21:39:45.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.268+0000 7fc6ce5ff700 1 -- start start 2023-07-06T21:39:45.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.269+0000 7fc6ce5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 0x7fc6c80b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.269+0000 7fc6ce5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c80c6670 0x7fc6c80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.269+0000 7fc6ce5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cc340 0x7fc6c80cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.269+0000 7fc6ce5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80c8110 con 0x7fc6b0003f80 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.269+0000 7fc6ce5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80c7ae0 con 0x7fc6c80c6670 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.269+0000 7fc6ce5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80c7020 con 0x7fc6c80cc340 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 0x7fc6c80b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 0x7fc6c80b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36070/0 (socket says 172.21.15.84:36070) 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 -- 172.21.15.84:0/3979904418 learned_addr learned my addr 172.21.15.84:0/3979904418 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:45.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c80c6670 0x7fc6c80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6ccb9c700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cc340 0x7fc6c80cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 -- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cc340 msgr2=0x7fc6c80cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cc340 0x7fc6c80cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 -- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c80c6670 msgr2=0x7fc6c80c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c80c6670 0x7fc6c80c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 -- 172.21.15.84:0/3979904418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c815d680 con 0x7fc6b0003f80 2023-07-06T21:39:45.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c80c6670 0x7fc6c80c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c7fff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 0x7fc6c80b6c30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8001390 tx=0x7fc6b800edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b8005e20 con 0x7fc6b0003f80 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b800c7e0 con 0x7fc6b0003f80 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.270+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c80cccf0 con 0x7fc6b0003f80 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.271+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b800cb50 con 0x7fc6b0003f80 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.271+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c80c7560 con 0x7fc6b0003f80 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.272+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c815b240 con 0x7fc6b0003f80 2023-07-06T21:39:45.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.272+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc6b800ccb0 con 0x7fc6b0003f80 2023-07-06T21:39:45.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.273+0000 7fc6c4ff9700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6b406ace0 0x7fc6b406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:45.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.273+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6b8089100 con 0x7fc6b0003f80 2023-07-06T21:39:45.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.274+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6b406ace0 0x7fc6b406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:45.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.274+0000 7fc6c77fe700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6b406ace0 0x7fc6b406d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0090b0 tx=0x7fc6bc01f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:45.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.278+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc6b80541b0 con 0x7fc6b0003f80 2023-07-06T21:39:45.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.514+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc6c80cd910 con 0x7fc6b0003f80 2023-07-06T21:39:45.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.515+0000 7fc6c4ff9700 1 -- 172.21.15.84:0/3979904418 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc6c80cd910 con 0x7fc6b0003f80 2023-07-06T21:39:45.518 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:39:45.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6b406ace0 msgr2=0x7fc6b406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:45.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6b406ace0 0x7fc6b406d190 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0090b0 tx=0x7fc6bc01f040 comp rx=0 tx=0).stop 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 msgr2=0x7fc6c80b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 0x7fc6c80b6c30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8001390 tx=0x7fc6b800edd0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 shutdown_connections 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc6b406ace0 0x7fc6b406d190 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6c80cc340 0x7fc6c80cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6b0003f80 0x7fc6c80b6c30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 --2- 172.21.15.84:0/3979904418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6c80c6670 0x7fc6c80c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:45.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 >> 172.21.15.84:0/3979904418 conn(0x7fc6c806b350 msgr2=0x7fc6c80b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:45.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 shutdown_connections 2023-07-06T21:39:45.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:45.517+0000 7fc6ce5ff700 1 -- 172.21.15.84:0/3979904418 wait complete. 2023-07-06T21:39:45.538 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:39:45.538 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:39:45.538 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:39:45.538 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:39:45.539 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:45.539 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:39:45.539 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:39:45.539 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:39:45.539 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:39:45.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:45.540 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:39:45.582 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:45.583 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:45.583 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:45.583 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:45.583 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:45.583 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:39:45.584 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42217sec preferred_lft 42217sec 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:39:45.585 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:45.586 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T21:39:45.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:45.587 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:45.587 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:45.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2023-07-06T21:39:45.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2023-07-06T21:39:45.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2023-07-06T21:39:45.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T21:39:45.589 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:46.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:46.051 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:39:46.121 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:39:46.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:46.125 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:39:46.193 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:39:46.193 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:39:46.193 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:39:46.193 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:39:46.193 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:39:46.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:46.195 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:46.195 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:46.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:39:46.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:39:46.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:39:46.196 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:46.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:46.848 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:39:46.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:46.892 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2023-07-06T21:39:46.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:46.975 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:46.975 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:46.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:39:46.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-07-06T21:39:46.976 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:47.155 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-07-06T21:39:47.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:47.156 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:47.156 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:47.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-07-06T21:39:47.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T21:39:47.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-07-06T21:39:47.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-07-06T21:39:47.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-07-06T21:39:47.158 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:47.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:47.421 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:47.421 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:47.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:39:47.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T21:39:47.423 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:47.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:39:47.601 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:39:47.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:47.602 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:47.645 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:39:47.688 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:39:47.688 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:47.689 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:47.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:39:47.878 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:47.879 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:47.945 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:47.947 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:47.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:47.949 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:39:47.993 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:39:47.993 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:39:48.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:39:48.107 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:48.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:48.174 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:48.176 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:48.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:48.177 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:39:48.229 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:39:48.227+0000 7fc6f62f45c0 -1 init, newargv = 0x55e4bda4ce90 newargc=15 2023-07-06T21:39:48.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[140102]: starting ceph client 2023-07-06T21:39:48.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[140102]: starting fuse 2023-07-06T21:39:49.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:39:49.400 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:49.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:49.470 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:49.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:49.472 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:39:49.515 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:39:49.515 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:39:49.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:39:49.516 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:39:49.516 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:39:49.516 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:39:49.516 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:39:49.517 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:49.517 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:39:49.517 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:39:49.517 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:39:49.518 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:39:49.518 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:49.518 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:39:49.518 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:39:49.519 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:39:49.519 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:39:49.519 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:49.519 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:39:49.520 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:39:49.520 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:39:49.520 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:39:49.520 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:39:49.521 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:39:49.521 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:39:49.521 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:39:49.521 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:49.522 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:39:49.522 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:39:49.695 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.140102.asok 2023-07-06T21:39:49.874 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.140102.asok 2023-07-06T21:39:49.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:49.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.140102.asok status 2023-07-06T21:39:50.098 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:39:50.098 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:39:50.098 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:39:50.098 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "140102", 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:39:50.099 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4668, 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 4668 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:50.100 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 4210843170 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:39:50.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 4210843170 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.4668 192.168.0.1:0/4210843170", 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/4210843170", 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 8, 2023-07-06T21:39:50.102 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 23, 2023-07-06T21:39:50.103 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:39:50.103 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:39:50.103 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:39:50.103 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:39:50.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:39:50.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.304+0000 7f9c879ff700 1 Processor -- start 2023-07-06T21:39:50.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.304+0000 7f9c879ff700 1 -- start start 2023-07-06T21:39:50.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c879ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9c800b83c0 0x7f9c800cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:50.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c879ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c800cfc60 0x7f9c800d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:50.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c879ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c800d0a80 con 0x7f9c800d3710 2023-07-06T21:39:50.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c879ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c800b8790 con 0x7f9c800cfc60 2023-07-06T21:39:50.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c879ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c800b4e70 con 0x7f9c800b83c0 2023-07-06T21:39:50.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c85f9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c800cfc60 0x7f9c800d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:50.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c85f9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9c800cfc60 0x7f9c800d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36078/0 (socket says 172.21.15.84:36078) 2023-07-06T21:39:50.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.305+0000 7f9c85f9c700 1 -- 172.21.15.84:0/897903215 learned_addr learned my addr 172.21.15.84:0/897903215 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:50.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c8579b700 1 --2- 172.21.15.84:0/897903215 >> v2:172.21.15.107:3300/0 conn(0x7f9c800b83c0 0x7f9c800cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c85f9c700 1 -- 172.21.15.84:0/897903215 >> v2:172.21.15.107:3300/0 conn(0x7f9c800b83c0 msgr2=0x7f9c800cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c85f9c700 1 --2- 172.21.15.84:0/897903215 >> v2:172.21.15.107:3300/0 conn(0x7f9c800b83c0 0x7f9c800cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c85f9c700 1 -- 172.21.15.84:0/897903215 >> v1:172.21.15.107:6789/0 conn(0x7f9c800d3710 legacy=0x7f9c800cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c85f9c700 1 -- 172.21.15.84:0/897903215 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800b5150 con 0x7f9c800cfc60 2023-07-06T21:39:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c85f9c700 1 --2- 172.21.15.84:0/897903215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800cfc60 0x7f9c800d0f10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c800cf520 tx=0x7f9c7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b61a0015f43b2b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c77fff700 1 -- 172.21.15.84:0/897903215 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800e070 con 0x7f9c800cfc60 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c77fff700 1 -- 172.21.15.84:0/897903215 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c78003a20 con 0x7f9c800cfc60 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.306+0000 7f9c77fff700 1 -- 172.21.15.84:0/897903215 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c78003dd0 con 0x7f9c800cfc60 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 -- 172.21.15.84:0/897903215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800cfc60 msgr2=0x7f9c800d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 --2- 172.21.15.84:0/897903215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800cfc60 0x7f9c800d0f10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9c800cf520 tx=0x7f9c7800b330 comp rx=0 tx=0).stop 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 -- 172.21.15.84:0/897903215 shutdown_connections 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 --2- 172.21.15.84:0/897903215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800cfc60 0x7f9c800d0f10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 --2- 172.21.15.84:0/897903215 >> v2:172.21.15.107:3300/0 conn(0x7f9c800b83c0 0x7f9c800cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 -- 172.21.15.84:0/897903215 >> 172.21.15.84:0/897903215 conn(0x7f9c8006b350 msgr2=0x7f9c8006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:50.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 -- 172.21.15.84:0/897903215 shutdown_connections 2023-07-06T21:39:50.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.307+0000 7f9c879ff700 1 -- 172.21.15.84:0/897903215 wait complete. 2023-07-06T21:39:50.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.309+0000 7f9c879ff700 1 Processor -- start 2023-07-06T21:39:50.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.309+0000 7f9c879ff700 1 -- start start 2023-07-06T21:39:50.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c879ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 0x7f9c800d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:50.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c879ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c800cfc60 0x7f9c8014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c879ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c800d3710 0x7f9c8014c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c879ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800ccc10 con 0x7f9c800b83c0 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c879ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800ccef0 con 0x7f9c800cfc60 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c879ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800cd1d0 con 0x7f9c800d3710 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c8579b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 0x7f9c800d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c8579b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 0x7f9c800d1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36084/0 (socket says 172.21.15.84:36084) 2023-07-06T21:39:50.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c8579b700 1 -- 172.21.15.84:0/1857918088 learned_addr learned my addr 172.21.15.84:0/1857918088 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c85f9c700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c800d3710 0x7f9c8014c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.310+0000 7f9c84f9a700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c800cfc60 0x7f9c8014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c8579b700 1 -- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c800d3710 msgr2=0x7f9c8014c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c8579b700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c800d3710 0x7f9c8014c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c8579b700 1 -- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c800cfc60 msgr2=0x7f9c8014bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c8579b700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c800cfc60 0x7f9c8014bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c8579b700 1 -- 172.21.15.84:0/1857918088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78009b30 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c8579b700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 0x7f9c800d1b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d800 tx=0x7f9c7c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c0125f0 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c012750 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8015bcc0 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c012b00 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.311+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c800d2090 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.312+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c800d2280 con 0x7f9c800b83c0 2023-07-06T21:39:50.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.313+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9c7c012c60 con 0x7f9c800b83c0 2023-07-06T21:39:50.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.314+0000 7f9c75ffb700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9c6406ace0 0x7f9c6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:50.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.314+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9c7c089510 con 0x7f9c800b83c0 2023-07-06T21:39:50.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.314+0000 7f9c84f9a700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9c6406ace0 0x7f9c6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:50.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.315+0000 7f9c84f9a700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9c6406ace0 0x7f9c6406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c800b2a70 tx=0x7f9c70007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:50.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.318+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9c7c054520 con 0x7f9c800b83c0 2023-07-06T21:39:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.561+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c800d13c0 con 0x7f9c800b83c0 2023-07-06T21:39:50.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.561+0000 7f9c75ffb700 1 -- 172.21.15.84:0/1857918088 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9c7c057b40 con 0x7f9c800b83c0 2023-07-06T21:39:50.564 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9c6406ace0 msgr2=0x7f9c6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9c6406ace0 0x7f9c6406d190 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9c800b2a70 tx=0x7f9c70007ec0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 msgr2=0x7f9c800d1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 0x7f9c800d1b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00d800 tx=0x7f9c7c00db10 comp rx=0 tx=0).stop 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 shutdown_connections 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9c6406ace0 0x7f9c6406d190 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c800d3710 0x7f9c8014c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.563+0000 7f9c879ff700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9c800b83c0 0x7f9c800d1b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.564+0000 7f9c879ff700 1 --2- 172.21.15.84:0/1857918088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c800cfc60 0x7f9c8014bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.564+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 >> 172.21.15.84:0/1857918088 conn(0x7f9c8006b350 msgr2=0x7f9c800cf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.564+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 shutdown_connections 2023-07-06T21:39:50.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:50.564+0000 7f9c879ff700 1 -- 172.21.15.84:0/1857918088 wait complete. 2023-07-06T21:39:50.583 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:39:50.584 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:39:50.584 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:39:50.584 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-07-06T21:39:50.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:50.585 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:39:50.585 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:39:50.585 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:39:50.585 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-07-06T21:39:50.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:50.586 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:39:50.629 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:50.629 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:50.630 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:50.630 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:50.630 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:50.630 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:50.630 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:50.630 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:50.631 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:39:50.631 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:50.631 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:39:50.632 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42211sec preferred_lft 42211sec 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:50.633 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:39:50.634 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:50.634 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:39:50.634 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:39:50.634 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:39:50.634 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:50.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:50.635 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:39:50.735 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:39:50.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:50.738 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:39:50.783 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:39:50.783 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:39:50.783 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:39:50.783 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:39:50.783 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:39:50.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:50.785 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:50.785 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:50.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:39:50.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:39:50.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:39:50.786 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:50.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:50.992 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:39:51.034 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:39:51.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:51.035 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2023-07-06T21:39:51.118 INFO:teuthology.orchestra.run.smithi084.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-07-06T21:39:51.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:51.120 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:51.120 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:51.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:39:51.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-07-06T21:39:51.121 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:51.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:51.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T21:39:51.368 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.369 INFO:teuthology.orchestra.run.smithi084.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:39:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 9a:50:c7:45:32:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T21:39:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T21:39:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::9850:c7ff:fe45:32f5/64 scope link 2023-07-06T21:39:51.370 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:51.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-07-06T21:39:51.439 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:51.439 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:51.439 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:51.439 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.440 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:51.440 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.440 INFO:teuthology.orchestra.run.smithi084.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:39:51.440 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 9a:50:c7:45:32:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T21:39:51.440 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T21:39:51.440 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.441 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::9850:c7ff:fe45:32f5/64 scope link 2023-07-06T21:39:51.441 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:51.442 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-07-06T21:39:51.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:51.443 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:51.443 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:51.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-07-06T21:39:51.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T21:39:51.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-07-06T21:39:51.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-07-06T21:39:51.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-07-06T21:39:51.444 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:51.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:51.687 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:39:51.688 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:39:51.688 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.1 up 2023-07-06T21:39:51.688 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T21:39:51.688 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:39:51.866 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-07-06T21:39:51.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:51.867 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:51.909 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-07-06T21:39:51.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:51.910 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:51.991 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:39:52.101 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:52.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:52.169 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:52.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:52.170 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:39:52.213 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:39:52.213 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2023-07-06T21:39:52.213 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-07-06T21:39:52.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:39:52.324 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:52.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:52.390 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:52.394 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:52.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:52.395 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:39:52.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi084.stderr:2023-07-06T21:39:52.421+0000 7f132b86a5c0 -1 init, newargv = 0x5585a0010db0 newargc=15 2023-07-06T21:39:52.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi084.stderr:ceph-fuse[140809]: starting ceph client 2023-07-06T21:39:52.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi084.stderr:ceph-fuse[140809]: starting fuse 2023-07-06T21:39:52.445 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:39:52.445 INFO:teuthology.orchestra.run.smithi084.stdout:44 2023-07-06T21:39:52.445 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2023-07-06T21:39:52.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:39:52.446 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:39:52.446 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:39:52.446 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:39:52.446 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:39:52.447 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:52.447 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:39:52.447 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-07-06T21:39:52.447 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:39:52.448 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-07-06T21:39:52.448 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:52.448 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:39:52.448 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:39:52.449 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:39:52.449 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:39:52.449 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:52.449 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:39:52.449 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:39:52.450 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:39:52.450 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:39:52.450 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:39:52.450 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:39:52.450 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:39:52.451 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:39:52.451 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:39:52.451 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.2")) 2023-07-06T21:39:52.451 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:39:52.626 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.2.140809.asok 2023-07-06T21:39:52.804 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.140809.asok 2023-07-06T21:39:52.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:52.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.140809.asok status 2023-07-06T21:39:53.027 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:39:53.027 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:39:53.027 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:39:53.028 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:39:53.028 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "2", 2023-07-06T21:39:53.028 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:39:53.028 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-07-06T21:39:53.028 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "140809", 2023-07-06T21:39:53.028 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 4677, 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:39:53.029 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 4677 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.2:0", 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1860593536 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:39:53.030 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.2:0", 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1860593536 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.4677 192.168.0.2:0/1860593536", 2023-07-06T21:39:53.031 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.2:0/1860593536", 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 8, 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 23, 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:39:53.032 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:39:53.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:39:53.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.233+0000 7fec90bff700 1 Processor -- start 2023-07-06T21:39:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.234+0000 7fec90bff700 1 -- start start 2023-07-06T21:39:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec90bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec8c0c9410 0x7fec8c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec90bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c070230 con 0x7fec8c15a7e0 2023-07-06T21:39:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec90bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c15b2a0 con 0x7fec8c154910 2023-07-06T21:39:53.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec90bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8c071390 con 0x7fec8c0c9410 2023-07-06T21:39:53.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec83fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fec8c15a7e0 0x7fec8c0c8d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56174/0 (socket says 172.21.15.84:56174) 2023-07-06T21:39:53.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec83fff700 1 -- 172.21.15.84:0/1738918530 learned_addr learned my addr 172.21.15.84:0/1738918530 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:39:53.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec8ad9d700 1 --2- 172.21.15.84:0/1738918530 >> v2:172.21.15.107:3300/0 conn(0x7fec8c0c9410 0x7fec8c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:53.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.235+0000 7fec89d9b700 1 -- 172.21.15.84:0/1738918530 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787695061 0 0) 0x7fec8c070230 con 0x7fec8c15a7e0 2023-07-06T21:39:53.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec89d9b700 1 -- 172.21.15.84:0/1738918530 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec780035b0 con 0x7fec8c15a7e0 2023-07-06T21:39:53.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec8ad9d700 1 -- 172.21.15.84:0/1738918530 >> v1:172.21.15.107:6789/0 conn(0x7fec8c154910 legacy=0x7fec8c070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec89d9b700 1 -- 172.21.15.84:0/1738918530 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262025915 0 0) 0x7fec8c15b2a0 con 0x7fec8c154910 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec8ad9d700 1 -- 172.21.15.84:0/1738918530 >> v1:172.21.15.84:6789/0 conn(0x7fec8c15a7e0 legacy=0x7fec8c0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec8ad9d700 1 -- 172.21.15.84:0/1738918530 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c15b580 con 0x7fec8c0c9410 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec8ad9d700 1 --2- 172.21.15.84:0/1738918530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c0c9410 0x7fec8c15ac30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec84009d90 tx=0x7fec8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8589d2aa20a637a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec89d9b700 1 -- 172.21.15.84:0/1738918530 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8400e070 con 0x7fec8c0c9410 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec89d9b700 1 -- 172.21.15.84:0/1738918530 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec84003a20 con 0x7fec8c0c9410 2023-07-06T21:39:53.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.236+0000 7fec89d9b700 1 -- 172.21.15.84:0/1738918530 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74001670 con 0x7fec8c0c9410 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 -- 172.21.15.84:0/1738918530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c0c9410 msgr2=0x7fec8c15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 --2- 172.21.15.84:0/1738918530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c0c9410 0x7fec8c15ac30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec84009d90 tx=0x7fec8400b330 comp rx=0 tx=0).stop 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 -- 172.21.15.84:0/1738918530 shutdown_connections 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 --2- 172.21.15.84:0/1738918530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c0c9410 0x7fec8c15ac30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 -- 172.21.15.84:0/1738918530 >> 172.21.15.84:0/1738918530 conn(0x7fec8c0cf3d0 msgr2=0x7fec8c0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 -- 172.21.15.84:0/1738918530 shutdown_connections 2023-07-06T21:39:53.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.237+0000 7fec90bff700 1 -- 172.21.15.84:0/1738918530 wait complete. 2023-07-06T21:39:53.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.239+0000 7fec90bff700 1 Processor -- start 2023-07-06T21:39:53.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.239+0000 7fec90bff700 1 -- start start 2023-07-06T21:39:53.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec90bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 0x7fec8c14d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:53.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec90bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c154910 0x7fec8c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:53.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec90bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8c15a7e0 0x7fec8c0b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec90bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0c82d0 con 0x7fec8c0c9410 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec90bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0b44c0 con 0x7fec8c154910 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec90bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c14a4c0 con 0x7fec8c15a7e0 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 0x7fec8c14d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 0x7fec8c14d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46820/0 (socket says 172.21.15.84:46820) 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 -- 172.21.15.84:0/1712583249 learned_addr learned my addr 172.21.15.84:0/1712583249 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:53.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8ad9d700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8c15a7e0 0x7fec8c0b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec83fff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c154910 0x7fec8c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 -- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8c15a7e0 msgr2=0x7fec8c0b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8c15a7e0 0x7fec8c0b2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 -- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c154910 msgr2=0x7fec8c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c154910 0x7fec8c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec8a59c700 1 -- 172.21.15.84:0/1712583249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec84009a70 con 0x7fec8c0c9410 2023-07-06T21:39:53.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.240+0000 7fec83fff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c154910 0x7fec8c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:39:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.241+0000 7fec8a59c700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 0x7fec8c14d180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c520 tx=0x7fec7c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.241+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0054e0 con 0x7fec8c0c9410 2023-07-06T21:39:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.241+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c003680 con 0x7fec8c0c9410 2023-07-06T21:39:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.241+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c14d880 con 0x7fec8c0c9410 2023-07-06T21:39:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.241+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c0039f0 con 0x7fec8c0c9410 2023-07-06T21:39:53.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.241+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c0c7f60 con 0x7fec8c0c9410 2023-07-06T21:39:53.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.242+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c14dda0 con 0x7fec8c0c9410 2023-07-06T21:39:53.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.243+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec7c003b50 con 0x7fec8c0c9410 2023-07-06T21:39:53.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.244+0000 7fec82ffd700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec6c06ace0 0x7fec6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:53.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.244+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec7c0888f0 con 0x7fec8c0c9410 2023-07-06T21:39:53.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.245+0000 7fec83fff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec6c06ace0 0x7fec6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:53.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.245+0000 7fec83fff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec6c06ace0 0x7fec6c06d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0c58a0 tx=0x7fec74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:53.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.249+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec7c0539b0 con 0x7fec8c0c9410 2023-07-06T21:39:53.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.488+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fec8c0c4f20 con 0x7fec8c0c9410 2023-07-06T21:39:53.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.488+0000 7fec82ffd700 1 -- 172.21.15.84:0/1712583249 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fec7c017070 con 0x7fec8c0c9410 2023-07-06T21:39:53.490 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:39:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec6c06ace0 msgr2=0x7fec6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec6c06ace0 0x7fec6c06d190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0c58a0 tx=0x7fec74009040 comp rx=0 tx=0).stop 2023-07-06T21:39:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 msgr2=0x7fec8c14d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 0x7fec8c14d180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fec7c00c520 tx=0x7fec7c00c8e0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 shutdown_connections 2023-07-06T21:39:53.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec6c06ace0 0x7fec6c06d190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec8c15a7e0 0x7fec8c0b2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c0c9410 0x7fec8c14d180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 --2- 172.21.15.84:0/1712583249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec8c154910 0x7fec8c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 >> 172.21.15.84:0/1712583249 conn(0x7fec8c0cf3d0 msgr2=0x7fec8c06c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 shutdown_connections 2023-07-06T21:39:53.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:53.491+0000 7fec90bff700 1 -- 172.21.15.84:0/1712583249 wait complete. 2023-07-06T21:39:53.511 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:39:53.511 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:39:53.512 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-06T21:39:53.512 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-07-06T21:39:53.513 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-07-06T21:39:53.513 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:39:53.513 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:39:53.513 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:39:53.513 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-07-06T21:39:53.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:53.514 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-06T21:39:53.561 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2023-07-06T21:39:53.563 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:53.564 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:39:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:39:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42205sec preferred_lft 42205sec 2023-07-06T21:39:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ba93:8c80:5e3a:9404/64 scope link noprefixroute 2023-07-06T21:39:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:53.565 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-06T21:39:53.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:53.566 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:53.566 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:53.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2023-07-06T21:39:53.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2023-07-06T21:39:53.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2023-07-06T21:39:53.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-06T21:39:53.568 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:53.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:53.984 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:39:54.055 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-06T21:39:54.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:54.060 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-06T21:39:54.112 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-06T21:39:54.113 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:39:54.113 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:39:54.113 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:39:54.113 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:39:54.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:54.114 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:54.115 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:54.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:39:54.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:39:54.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:39:54.115 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:54.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:54.807 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-06T21:39:54.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:54.850 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2023-07-06T21:39:54.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:54.933 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:54.933 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:54.933 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T21:39:54.934 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-07-06T21:39:54.934 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:55.115 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-07-06T21:39:55.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:55.116 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:55.116 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:55.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-07-06T21:39:55.116 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-06T21:39:55.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-07-06T21:39:55.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-07-06T21:39:55.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-07-06T21:39:55.117 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:55.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:55.382 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:55.382 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:55.382 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2023-07-06T21:39:55.383 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2023-07-06T21:39:55.383 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:55.561 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-07-06T21:39:55.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:55.562 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-07-06T21:39:55.604 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-07-06T21:39:55.604 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:55.605 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-07-06T21:39:55.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2023-07-06T21:39:55.793 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:55.794 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:55.859 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:55.862 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:55.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:55.863 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2023-07-06T21:39:55.905 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:39:55.906 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-07-06T21:39:55.950 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2023-07-06T21:39:56.018 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:56.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:56.089 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:56.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:56.090 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2023-07-06T21:39:56.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi107.stderr:2023-07-06T21:39:56.122+0000 7f4ae46cc5c0 -1 init, newargv = 0x564c712821f0 newargc=15 2023-07-06T21:39:56.123 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi107.stderr:ceph-fuse[136852]: starting ceph client 2023-07-06T21:39:56.132 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi107.stderr:ceph-fuse[136852]: starting fuse 2023-07-06T21:39:56.138 INFO:teuthology.orchestra.run.smithi107.stdout:43 2023-07-06T21:39:56.138 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2023-07-06T21:39:56.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:39:56.139 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2023-07-06T21:39:56.139 DEBUG:teuthology.orchestra.run.smithi107:> import re 2023-07-06T21:39:56.139 DEBUG:teuthology.orchestra.run.smithi107:> import os 2023-07-06T21:39:56.140 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2023-07-06T21:39:56.140 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:56.140 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2023-07-06T21:39:56.140 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-07-06T21:39:56.141 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2023-07-06T21:39:56.141 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-07-06T21:39:56.141 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:56.141 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2023-07-06T21:39:56.141 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2023-07-06T21:39:56.142 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2023-07-06T21:39:56.142 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2023-07-06T21:39:56.142 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:56.142 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2023-07-06T21:39:56.143 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:39:56.143 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:39:56.143 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:39:56.143 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2023-07-06T21:39:56.143 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2023-07-06T21:39:56.144 DEBUG:teuthology.orchestra.run.smithi107:> return f 2023-07-06T21:39:56.144 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:39:56.144 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:56.144 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.1")) 2023-07-06T21:39:56.145 DEBUG:teuthology.orchestra.run.smithi107:> ' 2023-07-06T21:39:56.320 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.1.136852.asok 2023-07-06T21:39:56.498 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.136852.asok 2023-07-06T21:39:56.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:56.499 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.136852.asok status 2023-07-06T21:39:56.719 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-06T21:39:56.719 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2023-07-06T21:39:56.719 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:39:56.720 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:39:56.720 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "1", 2023-07-06T21:39:56.720 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2023-07-06T21:39:56.721 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-07-06T21:39:56.721 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "136852", 2023-07-06T21:39:56.721 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2023-07-06T21:39:56.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:56.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2023-07-06T21:39:56.722 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2023-07-06T21:39:56.722 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4669, 2023-07-06T21:39:56.722 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2023-07-06T21:39:56.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 4669 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1261930999 2023-07-06T21:39:56.723 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1261930999 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:56.724 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.4669 192.168.0.1:0/1261930999", 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/1261930999", 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 8, 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 23, 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2023-07-06T21:39:56.725 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-06T21:39:56.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:39:56.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.935+0000 7fe77dfff700 1 Processor -- start 2023-07-06T21:39:56.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.936+0000 7fe77dfff700 1 -- start start 2023-07-06T21:39:56.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe77dfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7780b83c0 0x7fe7780cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe77dfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780d3710 0x7fe7780cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe77dfff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe7780cfa90 0x7fe77815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe77dfff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780cfee0 con 0x7fe7780b83c0 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe77dfff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7780d0a80 con 0x7fe7780cfa90 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe77dfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780b8790 con 0x7fe7780d3710 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe7777fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7780b83c0 0x7fe7780cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe7777fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7780b83c0 0x7fe7780cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46832/0 (socket says 172.21.15.84:46832) 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe7777fe700 1 -- 172.21.15.84:0/87468857 learned_addr learned my addr 172.21.15.84:0/87468857 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe776ffd700 1 --2- 172.21.15.84:0/87468857 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780d3710 0x7fe7780cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe777fff700 1 --2- 172.21.15.84:0/87468857 >> v2:172.21.15.107:3300/0 conn(0x7fe7780cfa90 0x7fe77815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.937+0000 7fe7777fe700 1 -- 172.21.15.84:0/87468857 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780d3710 msgr2=0x7fe7780cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7777fe700 1 --2- 172.21.15.84:0/87468857 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780d3710 0x7fe7780cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7777fe700 1 -- 172.21.15.84:0/87468857 >> v2:172.21.15.107:3300/0 conn(0x7fe7780cfa90 msgr2=0x7fe77815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7777fe700 1 --2- 172.21.15.84:0/87468857 >> v2:172.21.15.107:3300/0 conn(0x7fe7780cfa90 0x7fe77815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7777fe700 1 -- 172.21.15.84:0/87468857 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780b4e70 con 0x7fe7780b83c0 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7777fe700 1 --2- 172.21.15.84:0/87468857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780b83c0 0x7fe7780cd200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7780ce5b0 tx=0x7fe768009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e175f616890a5f68 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:56.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7767fc700 1 -- 172.21.15.84:0/87468857 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768011830 con 0x7fe7780b83c0 2023-07-06T21:39:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7767fc700 1 -- 172.21.15.84:0/87468857 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe768011990 con 0x7fe7780b83c0 2023-07-06T21:39:56.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.938+0000 7fe7767fc700 1 -- 172.21.15.84:0/87468857 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768011d20 con 0x7fe7780b83c0 2023-07-06T21:39:56.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 -- 172.21.15.84:0/87468857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780b83c0 msgr2=0x7fe7780cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 --2- 172.21.15.84:0/87468857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780b83c0 0x7fe7780cd200 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe7780ce5b0 tx=0x7fe768009bd0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 -- 172.21.15.84:0/87468857 shutdown_connections 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 --2- 172.21.15.84:0/87468857 >> v2:172.21.15.107:3300/0 conn(0x7fe7780cfa90 0x7fe77815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 --2- 172.21.15.84:0/87468857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780b83c0 0x7fe7780cd200 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 --2- 172.21.15.84:0/87468857 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780d3710 0x7fe7780cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 -- 172.21.15.84:0/87468857 >> 172.21.15.84:0/87468857 conn(0x7fe77806b350 msgr2=0x7fe77806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:56.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 -- 172.21.15.84:0/87468857 shutdown_connections 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.939+0000 7fe77dfff700 1 -- 172.21.15.84:0/87468857 wait complete. 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.940+0000 7fe77dfff700 1 Processor -- start 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.941+0000 7fe77dfff700 1 -- start start 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.941+0000 7fe77dfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780b83c0 0x7fe77815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe77dfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7780cfa90 0x7fe7780c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe77dfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 0x7fe7780c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe77dfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781f2110 con 0x7fe7780d3710 2023-07-06T21:39:56.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe77dfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7781f2eb0 con 0x7fe7780cfa90 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe77dfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780c9480 con 0x7fe7780b83c0 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe777fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 0x7fe7780c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe777fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 0x7fe7780c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46842/0 (socket says 172.21.15.84:46842) 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe7777fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780b83c0 0x7fe77815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe777fff700 1 -- 172.21.15.84:0/257506846 learned_addr learned my addr 172.21.15.84:0/257506846 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe776ffd700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7780cfa90 0x7fe7780c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe7777fe700 1 -- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7780cfa90 msgr2=0x7fe7780c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:56.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe7777fe700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7780cfa90 0x7fe7780c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe7777fe700 1 -- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 msgr2=0x7fe7780c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:56.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe7777fe700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 0x7fe7780c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:56.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe7777fe700 1 -- 172.21.15.84:0/257506846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe768009550 con 0x7fe7780b83c0 2023-07-06T21:39:56.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.942+0000 7fe777fff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 0x7fe7780c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:39:56.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.943+0000 7fe7777fe700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780b83c0 0x7fe77815adb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe768016000 tx=0x7fe768004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:56.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.943+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe768004510 con 0x7fe7780b83c0 2023-07-06T21:39:56.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.943+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76801e070 con 0x7fe7780b83c0 2023-07-06T21:39:56.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.943+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7780c9760 con 0x7fe7780b83c0 2023-07-06T21:39:56.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.943+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7680038e0 con 0x7fe7780b83c0 2023-07-06T21:39:56.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.943+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7781f33a0 con 0x7fe7780b83c0 2023-07-06T21:39:56.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.944+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778073b30 con 0x7fe7780b83c0 2023-07-06T21:39:56.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.945+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe768003a40 con 0x7fe7780b83c0 2023-07-06T21:39:56.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.946+0000 7fe75ffff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe76406ace0 0x7fe76406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:39:56.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.946+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe76808a200 con 0x7fe7780b83c0 2023-07-06T21:39:56.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.946+0000 7fe776ffd700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe76406ace0 0x7fe76406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:39:56.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.946+0000 7fe776ffd700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe76406ace0 0x7fe76406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7781f2e60 tx=0x7fe76c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:39:56.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:56.950+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe7680552a0 con 0x7fe7780b83c0 2023-07-06T21:39:57.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.191+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe7780c98f0 con 0x7fe7780b83c0 2023-07-06T21:39:57.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.191+0000 7fe75ffff700 1 -- 172.21.15.84:0/257506846 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe7680588c0 con 0x7fe7780b83c0 2023-07-06T21:39:57.193 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:39:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe76406ace0 msgr2=0x7fe76406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe76406ace0 0x7fe76406d190 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7781f2e60 tx=0x7fe76c009210 comp rx=0 tx=0).stop 2023-07-06T21:39:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780b83c0 msgr2=0x7fe77815adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780b83c0 0x7fe77815adb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe768016000 tx=0x7fe768004750 comp rx=0 tx=0).stop 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 shutdown_connections 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe76406ace0 0x7fe76406d190 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7780d3710 0x7fe7780c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7780b83c0 0x7fe77815adb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 --2- 172.21.15.84:0/257506846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7780cfa90 0x7fe7780c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 >> 172.21.15.84:0/257506846 conn(0x7fe77806b350 msgr2=0x7fe7780b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:39:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 shutdown_connections 2023-07-06T21:39:57.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:39:57.194+0000 7fe77dfff700 1 -- 172.21.15.84:0/257506846 wait complete. 2023-07-06T21:39:57.214 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:39:57.214 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:39:57.214 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2023-07-06T21:39:57.215 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-07-06T21:39:57.215 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-07-06T21:39:57.215 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:39:57.215 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:39:57.215 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:39:57.215 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-07-06T21:39:57.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.216 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2023-07-06T21:39:57.260 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:57.260 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2023-07-06T21:39:57.261 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:39:57.262 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42201sec preferred_lft 42201sec 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ba93:8c80:5e3a:9404/64 scope link noprefixroute 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether de:34:75:19:21:9d brd ff:ff:ff:ff:ff:ff 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.263 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::1031:a2ff:fe1e:e7d/64 scope link 2023-07-06T21:39:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.264 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:39:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether de:34:75:19:21:9d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T21:39:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::dc34:75ff:fe19:219d/64 scope link 2023-07-06T21:39:57.264 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.265 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:39:57.366 INFO:teuthology.orchestra.run.smithi107.stdout:1 2023-07-06T21:39:57.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.368 DEBUG:teuthology.orchestra.run.smithi107:> route 2023-07-06T21:39:57.414 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2023-07-06T21:39:57.415 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:39:57.415 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:39:57.415 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:39:57.416 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:39:57.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.417 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:57.417 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:57.417 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:39:57.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:39:57.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:39:57.419 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:57.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.621 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-06T21:39:57.663 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T21:39:57.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.664 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2023-07-06T21:39:57.747 INFO:teuthology.orchestra.run.smithi107.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-07-06T21:39:57.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.748 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:57.748 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:57.749 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T21:39:57.749 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-07-06T21:39:57.749 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:57.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T21:39:57.996 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.996 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 2e:1a:74:bb:4a:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::2c1a:74ff:febb:4af1/64 scope link 2023-07-06T21:39:57.997 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:57.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:57.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-07-06T21:39:58.067 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:39:58.067 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:39:58.067 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:39:58.067 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:58.067 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2023-07-06T21:39:58.067 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:58.068 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:39:58.068 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 2e:1a:74:bb:4a:f1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-07-06T21:39:58.068 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-07-06T21:39:58.068 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:58.068 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::2c1a:74ff:febb:4af1/64 scope link 2023-07-06T21:39:58.068 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:39:58.069 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-07-06T21:39:58.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:58.070 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:58.070 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:58.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-07-06T21:39:58.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-07-06T21:39:58.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-07-06T21:39:58.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-07-06T21:39:58.071 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-07-06T21:39:58.072 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:58.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:58.320 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2023-07-06T21:39:58.320 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2023-07-06T21:39:58.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.1 up 2023-07-06T21:39:58.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.1 master ceph-brx 2023-07-06T21:39:58.321 DEBUG:teuthology.orchestra.run.smithi107:> ') 2023-07-06T21:39:58.499 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-07-06T21:39:58.499 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:58.500 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-07-06T21:39:58.543 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-07-06T21:39:58.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:39:58.544 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-07-06T21:39:58.625 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2023-07-06T21:39:58.733 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:58.733 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:58.798 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:58.800 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:58.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:58.801 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2023-07-06T21:39:58.843 INFO:teuthology.orchestra.run.smithi107.stdout:43 2023-07-06T21:39:58.843 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2023-07-06T21:39:58.844 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-07-06T21:39:58.888 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2023-07-06T21:39:58.955 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:39:58.956 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:39:59.023 INFO:teuthology.orchestra.run.smithi107.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:39:59.027 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:39:59.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.028 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2023-07-06T21:39:59.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi107.stderr:2023-07-06T21:39:59.053+0000 7f5c8a3065c0 -1 init, newargv = 0x55e597eb4f80 newargc=15 2023-07-06T21:39:59.055 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi107.stderr:ceph-fuse[137460]: starting ceph client 2023-07-06T21:39:59.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi107.stderr:ceph-fuse[137460]: starting fuse 2023-07-06T21:39:59.074 INFO:teuthology.orchestra.run.smithi107.stdout:43 2023-07-06T21:39:59.074 INFO:teuthology.orchestra.run.smithi107.stdout:44 2023-07-06T21:39:59.074 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2023-07-06T21:39:59.075 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:39:59.075 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2023-07-06T21:39:59.075 DEBUG:teuthology.orchestra.run.smithi107:> import re 2023-07-06T21:39:59.076 DEBUG:teuthology.orchestra.run.smithi107:> import os 2023-07-06T21:39:59.076 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2023-07-06T21:39:59.076 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:59.076 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2023-07-06T21:39:59.077 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-07-06T21:39:59.077 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2023-07-06T21:39:59.077 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-07-06T21:39:59.077 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:59.077 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2023-07-06T21:39:59.078 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2023-07-06T21:39:59.078 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2023-07-06T21:39:59.078 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2023-07-06T21:39:59.078 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:59.079 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2023-07-06T21:39:59.079 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:39:59.079 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:39:59.079 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:39:59.079 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2023-07-06T21:39:59.080 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2023-07-06T21:39:59.080 DEBUG:teuthology.orchestra.run.smithi107:> return f 2023-07-06T21:39:59.080 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:39:59.081 DEBUG:teuthology.orchestra.run.smithi107:> 2023-07-06T21:39:59.081 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.3")) 2023-07-06T21:39:59.081 DEBUG:teuthology.orchestra.run.smithi107:> ' 2023-07-06T21:39:59.254 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.3.137460.asok 2023-07-06T21:39:59.432 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.137460.asok 2023-07-06T21:39:59.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.433 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.137460.asok status 2023-07-06T21:39:59.655 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2023-07-06T21:39:59.655 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "3", 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "137460", 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2023-07-06T21:39:59.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4544, 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2023-07-06T21:39:59.657 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 4544 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.2:0", 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4262825972 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: } 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:59.658 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.2:0", 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4262825972 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.4544 192.168.0.2:0/4262825972", 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.2:0/4262825972", 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2023-07-06T21:39:59.659 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 8, 2023-07-06T21:39:59.660 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 23, 2023-07-06T21:39:59.660 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:39:59.660 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2023-07-06T21:39:59.660 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2023-07-06T21:39:59.660 INFO:teuthology.orchestra.run.smithi107.stdout:} 2023-07-06T21:39:59.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.670 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:39:59.670 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:59.719 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:39:59.719 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:59.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:39:59.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.794 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:39:59.794 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:59.838 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:39:59.839 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:59.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-07-06T21:39:59.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.950 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2023-07-06T21:39:59.951 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.1 2023-07-06T21:39:59.994 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2023-07-06T21:39:59.995 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-07-06T21:39:59.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:39:59.995 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-07-06T21:40:00.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:00.114 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2023-07-06T21:40:00.114 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.3 2023-07-06T21:40:00.159 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2023-07-06T21:40:00.159 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-07-06T21:40:00.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:00.160 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-07-06T21:40:00.268 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-07-06T21:40:00.285 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.485+0000 7f85757ff700 1 Processor -- start 2023-07-06T21:40:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.485+0000 7f85757ff700 1 -- start start 2023-07-06T21:40:00.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f85757ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 0x7f8570070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f85757ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85700c9c10 0x7f85700ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f85757ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857015d930 con 0x7f8570159130 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f85757ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8570070bc0 con 0x7f85700c9c10 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f85757ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857015cd20 con 0x7f8570070230 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f8567fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 0x7f8570070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f856f7fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85700c9c10 0x7f85700ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f8567fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 0x7f8570070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:50424/0 (socket says 172.21.15.84:50424) 2023-07-06T21:40:00.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.487+0000 7f8567fff700 1 -- 172.21.15.84:0/973039656 learned_addr learned my addr 172.21.15.84:0/973039656 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856f7fe700 1 -- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 msgr2=0x7f8570070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856f7fe700 1 --2- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 0x7f8570070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856f7fe700 1 -- 172.21.15.84:0/973039656 >> v1:172.21.15.107:6789/0 conn(0x7f8570159130 legacy=0x7f85700c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856f7fe700 1 -- 172.21.15.84:0/973039656 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857015d000 con 0x7f85700c9c10 2023-07-06T21:40:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f8567fff700 1 --2- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 0x7f8570070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856e7fc700 1 -- 172.21.15.84:0/973039656 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988195808 0 0) 0x7f857015d930 con 0x7f8570159130 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856f7fe700 1 --2- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f85700ca030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d84fe0ed6896a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856e7fc700 1 -- 172.21.15.84:0/973039656 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856800e070 con 0x7f85700c9c10 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856e7fc700 1 -- 172.21.15.84:0/973039656 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856800bb20 con 0x7f85700c9c10 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.488+0000 7f856e7fc700 1 -- 172.21.15.84:0/973039656 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568012430 con 0x7f85700c9c10 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 -- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 msgr2=0x7f85700ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 --2- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f85700ca030 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 -- 172.21.15.84:0/973039656 shutdown_connections 2023-07-06T21:40:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 --2- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f85700ca030 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 --2- 172.21.15.84:0/973039656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570070230 0x7f8570070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 -- 172.21.15.84:0/973039656 >> 172.21.15.84:0/973039656 conn(0x7f85700cf3d0 msgr2=0x7f85700cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 -- 172.21.15.84:0/973039656 shutdown_connections 2023-07-06T21:40:00.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.489+0000 7f85757ff700 1 -- 172.21.15.84:0/973039656 wait complete. 2023-07-06T21:40:00.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.491+0000 7f85757ff700 1 Processor -- start 2023-07-06T21:40:00.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.491+0000 7f85757ff700 1 -- start start 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f85757ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 0x7f857015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f85757ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f857015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f85757ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570159130 0x7f857006d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f85757ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857015bdc0 con 0x7f8570070230 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f85757ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8570152c20 con 0x7f85700c9c10 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f85757ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f857015b1b0 con 0x7f8570159130 2023-07-06T21:40:00.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 0x7f857015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 0x7f857015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46856/0 (socket says 172.21.15.84:46856) 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 -- 172.21.15.84:0/2581167520 learned_addr learned my addr 172.21.15.84:0/2581167520 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f8567fff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f857015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856f7fe700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570159130 0x7f857006d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 -- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570159130 msgr2=0x7f857006d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570159130 0x7f857006d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 -- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 msgr2=0x7f857015c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f857015c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.492+0000 7f856effd700 1 -- 172.21.15.84:0/2581167520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8558009550 con 0x7f8570070230 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.493+0000 7f856effd700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 0x7f857015abd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f85600011e0 tx=0x7f856000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.493+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85600054e0 con 0x7f8570070230 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.493+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560003680 con 0x7f8570070230 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.493+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8568009a70 con 0x7f8570070230 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.493+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560003a30 con 0x7f8570070230 2023-07-06T21:40:00.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.493+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f857015b770 con 0x7f8570070230 2023-07-06T21:40:00.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.494+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8570158400 con 0x7f8570070230 2023-07-06T21:40:00.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.495+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8560003b90 con 0x7f8570070230 2023-07-06T21:40:00.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.496+0000 7f85677fe700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f855006ace0 0x7f855006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.496+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f85600888e0 con 0x7f8570070230 2023-07-06T21:40:00.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.496+0000 7f8567fff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f855006ace0 0x7f855006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.497+0000 7f8567fff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f855006ace0 0x7f855006d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85701f35b0 tx=0x7f85580092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:00.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.501+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8560053980 con 0x7f8570070230 2023-07-06T21:40:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.702+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85700ad820 con 0x7f8570070230 2023-07-06T21:40:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.703+0000 7f85677fe700 1 -- 172.21.15.84:0/2581167520 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7f8560017090 con 0x7f8570070230 2023-07-06T21:40:00.706 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:00.706 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f855006ace0 msgr2=0x7f855006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f855006ace0 0x7f855006d190 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f85701f35b0 tx=0x7f85580092c0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 msgr2=0x7f857015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 0x7f857015abd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f85600011e0 tx=0x7f856000ca30 comp rx=0 tx=0).stop 2023-07-06T21:40:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 shutdown_connections 2023-07-06T21:40:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f855006ace0 0x7f855006d190 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8570159130 0x7f857006d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8570070230 0x7f857015abd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 --2- 172.21.15.84:0/2581167520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85700c9c10 0x7f857015c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 >> 172.21.15.84:0/2581167520 conn(0x7f85700cf3d0 msgr2=0x7f85700cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 shutdown_connections 2023-07-06T21:40:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.706+0000 7f85757ff700 1 -- 172.21.15.84:0/2581167520 wait complete. 2023-07-06T21:40:00.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:00.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:00.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.922+0000 7fbd57bff700 1 Processor -- start 2023-07-06T21:40:00.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.922+0000 7fbd57bff700 1 -- start start 2023-07-06T21:40:00.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd57bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbd5015c020 0x7fbd500c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd57bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd5006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd57bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5006cf40 con 0x7fbd5015c460 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd57bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5015d400 con 0x7fbd5015c020 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd57bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5015ce40 con 0x7fbd5006a5a0 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd5599b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbd5015c020 0x7fbd500c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd5599b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbd5015c020 0x7fbd500c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46860/0 (socket says 172.21.15.84:46860) 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.924+0000 7fbd5599b700 1 -- 172.21.15.84:0/1178627826 learned_addr learned my addr 172.21.15.84:0/1178627826 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:00.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd5619c700 1 --2- 172.21.15.84:0/1178627826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd5006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd5599b700 1 -- 172.21.15.84:0/1178627826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 msgr2=0x7fbd5006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd5599b700 1 --2- 172.21.15.84:0/1178627826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd5006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd5599b700 1 -- 172.21.15.84:0/1178627826 >> v1:172.21.15.107:6789/0 conn(0x7fbd5015c460 legacy=0x7fbd50069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd5599b700 1 -- 172.21.15.84:0/1178627826 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5015d120 con 0x7fbd5015c020 2023-07-06T21:40:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd5599b700 1 --2- 172.21.15.84:0/1178627826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c020 0x7fbd500c90a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0098f0 tx=0x7fbd4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba66c4e7f87a6f74 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd54999700 1 -- 172.21.15.84:0/1178627826 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c011830 con 0x7fbd5015c020 2023-07-06T21:40:00.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd54999700 1 -- 172.21.15.84:0/1178627826 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c011990 con 0x7fbd5015c020 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.925+0000 7fbd54999700 1 -- 172.21.15.84:0/1178627826 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c011d20 con 0x7fbd5015c020 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 -- 172.21.15.84:0/1178627826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c020 msgr2=0x7fbd500c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1178627826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c020 0x7fbd500c90a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0098f0 tx=0x7fbd4c009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 -- 172.21.15.84:0/1178627826 shutdown_connections 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1178627826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd5006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1178627826 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c020 0x7fbd500c90a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 -- 172.21.15.84:0/1178627826 >> 172.21.15.84:0/1178627826 conn(0x7fbd500cbf10 msgr2=0x7fbd500ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:00.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 -- 172.21.15.84:0/1178627826 shutdown_connections 2023-07-06T21:40:00.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.926+0000 7fbd57bff700 1 -- 172.21.15.84:0/1178627826 wait complete. 2023-07-06T21:40:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.928+0000 7fbd57bff700 1 Processor -- start 2023-07-06T21:40:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.928+0000 7fbd57bff700 1 -- start start 2023-07-06T21:40:00.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd57bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd500c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd57bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd5015c020 0x7fbd5014eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd57bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 0x7fbd500c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd57bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500c66f0 con 0x7fbd5015c460 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd57bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5014f1e0 con 0x7fbd5015c020 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd57bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd500ab950 con 0x7fbd5006a5a0 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd5619c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 0x7fbd500c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd5619c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 0x7fbd500c2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46870/0 (socket says 172.21.15.84:46870) 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd5619c700 1 -- 172.21.15.84:0/1362121077 learned_addr learned my addr 172.21.15.84:0/1362121077 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd5599b700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd500c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.929+0000 7fbd5519a700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd5015c020 0x7fbd5014eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5619c700 1 -- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 msgr2=0x7fbd500c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5619c700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd500c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5619c700 1 -- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd5015c020 msgr2=0x7fbd5014eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:00.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5619c700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd5015c020 0x7fbd5014eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:00.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5619c700 1 -- 172.21.15.84:0/1362121077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c009550 con 0x7fbd5015c460 2023-07-06T21:40:00.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5519a700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd5015c020 0x7fbd5014eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:00.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd5619c700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 0x7fbd500c2b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800efc0 tx=0x7fbd48009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:00.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48009b20 con 0x7fbd5015c460 2023-07-06T21:40:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4800f040 con 0x7fbd5015c460 2023-07-06T21:40:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd500abc30 con 0x7fbd5015c460 2023-07-06T21:40:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800cd40 con 0x7fbd5015c460 2023-07-06T21:40:00.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.930+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd500ac1f0 con 0x7fbd5015c460 2023-07-06T21:40:00.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.931+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5004e1d0 con 0x7fbd5015c460 2023-07-06T21:40:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.932+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbd48014480 con 0x7fbd5015c460 2023-07-06T21:40:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.934+0000 7fbd467fc700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbd3406ace0 0x7fbd3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.934+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbd48089fc0 con 0x7fbd5015c460 2023-07-06T21:40:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.934+0000 7fbd5599b700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbd3406ace0 0x7fbd3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:00.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.934+0000 7fbd5599b700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbd3406ace0 0x7fbd3406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009c40 tx=0x7fbd4c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:00.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:00.938+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbd48054fd0 con 0x7fbd5015c460 2023-07-06T21:40:01.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.134+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd500ce2e0 con 0x7fbd5015c460 2023-07-06T21:40:01.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.134+0000 7fbd467fc700 1 -- 172.21.15.84:0/1362121077 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd480180a0 con 0x7fbd5015c460 2023-07-06T21:40:01.137 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:01.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbd3406ace0 msgr2=0x7fbd3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbd3406ace0 0x7fbd3406d190 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009c40 tx=0x7fbd4c008030 comp rx=0 tx=0).stop 2023-07-06T21:40:01.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 msgr2=0x7fbd500c2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 0x7fbd500c2b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800efc0 tx=0x7fbd48009f60 comp rx=0 tx=0).stop 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 shutdown_connections 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbd3406ace0 0x7fbd3406d190 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd5015c460 0x7fbd500c2b60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd5006a5a0 0x7fbd500c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 --2- 172.21.15.84:0/1362121077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd5015c020 0x7fbd5014eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 >> 172.21.15.84:0/1362121077 conn(0x7fbd500cbf10 msgr2=0x7fbd50069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 shutdown_connections 2023-07-06T21:40:01.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.137+0000 7fbd57bff700 1 -- 172.21.15.84:0/1362121077 wait complete. 2023-07-06T21:40:01.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:01.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.351+0000 7f17ca5ff700 1 Processor -- start 2023-07-06T21:40:01.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.352+0000 7f17ca5ff700 1 -- start start 2023-07-06T21:40:01.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17ca5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b81c0 0x7f17c40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17ca5ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17c40ceda0 0x7f17c40cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17ca5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17c40d1830 con 0x7f17c40b1a80 2023-07-06T21:40:01.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17ca5ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c40cf330 con 0x7f17c40ceda0 2023-07-06T21:40:01.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17ca5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40d1270 con 0x7f17c40b81c0 2023-07-06T21:40:01.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17c8b9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f17c40ceda0 0x7f17c40cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17bbfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f17c40b1a80 0x7f17c415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56184/0 (socket says 172.21.15.84:56184) 2023-07-06T21:40:01.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17bbfff700 1 -- 172.21.15.84:0/865039783 learned_addr learned my addr 172.21.15.84:0/865039783 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:01.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17c3fff700 1 --2- 172.21.15.84:0/865039783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b81c0 0x7f17c40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17c8b9c700 1 -- 172.21.15.84:0/865039783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b81c0 msgr2=0x7f17c40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.353+0000 7f17c8b9c700 1 --2- 172.21.15.84:0/865039783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b81c0 0x7f17c40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c8b9c700 1 -- 172.21.15.84:0/865039783 >> v1:172.21.15.84:6789/0 conn(0x7f17c40b1a80 legacy=0x7f17c415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c8b9c700 1 -- 172.21.15.84:0/865039783 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c40d1550 con 0x7f17c40ceda0 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c37fe700 1 -- 172.21.15.84:0/865039783 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790245256 0 0) 0x7f17c40d1830 con 0x7f17c40b1a80 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c8b9c700 1 --2- 172.21.15.84:0/865039783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cfb40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009d90 tx=0x7f17bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcd870b9d8805ff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c37fe700 1 -- 172.21.15.84:0/865039783 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00e070 con 0x7f17c40ceda0 2023-07-06T21:40:01.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c37fe700 1 -- 172.21.15.84:0/865039783 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00bbd0 con 0x7f17c40ceda0 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.354+0000 7f17c37fe700 1 -- 172.21.15.84:0/865039783 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc012430 con 0x7f17c40ceda0 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 -- 172.21.15.84:0/865039783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 msgr2=0x7f17c40cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/865039783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cfb40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009d90 tx=0x7f17bc00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 -- 172.21.15.84:0/865039783 shutdown_connections 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/865039783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cfb40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/865039783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b81c0 0x7f17c40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 -- 172.21.15.84:0/865039783 >> 172.21.15.84:0/865039783 conn(0x7f17c406b2b0 msgr2=0x7f17c406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:01.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 -- 172.21.15.84:0/865039783 shutdown_connections 2023-07-06T21:40:01.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.355+0000 7f17ca5ff700 1 -- 172.21.15.84:0/865039783 wait complete. 2023-07-06T21:40:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.356+0000 7f17ca5ff700 1 Processor -- start 2023-07-06T21:40:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.357+0000 7f17ca5ff700 1 -- start start 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.357+0000 7f17ca5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b1a80 0x7f17c41f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.357+0000 7f17ca5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40b81c0 0x7f17c40cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17ca5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17ca5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c41f1c80 con 0x7f17c40ceda0 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17ca5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c41e3e10 con 0x7f17c40b81c0 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17ca5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c40cb880 con 0x7f17c40b1a80 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46878/0 (socket says 172.21.15.84:46878) 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 -- 172.21.15.84:0/1190678214 learned_addr learned my addr 172.21.15.84:0/1190678214 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c3fff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b1a80 0x7f17c41f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17bbfff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40b81c0 0x7f17c40cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 -- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b1a80 msgr2=0x7f17c41f1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b1a80 0x7f17c41f1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 -- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40b81c0 msgr2=0x7f17c40cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40b81c0 0x7f17c40cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 -- 172.21.15.84:0/1190678214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc009a70 con 0x7f17c40ceda0 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17bbfff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40b81c0 0x7f17c40cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.358+0000 7f17c8b9c700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cb250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17c406bed0 tx=0x7f17bc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.359+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc012430 con 0x7f17c40ceda0 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.359+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc020070 con 0x7f17c40ceda0 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.359+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c40cbb60 con 0x7f17c40ceda0 2023-07-06T21:40:01.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.359+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004280 con 0x7f17c40ceda0 2023-07-06T21:40:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.359+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c40cc120 con 0x7f17c40ceda0 2023-07-06T21:40:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.360+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c40b4cf0 con 0x7f17c40ceda0 2023-07-06T21:40:01.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.361+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f17bc0043e0 con 0x7f17c40ceda0 2023-07-06T21:40:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.362+0000 7f17c17fa700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17a406ace0 0x7f17a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.362+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17bc022070 con 0x7f17c40ceda0 2023-07-06T21:40:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.362+0000 7f17c3fff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17a406ace0 0x7f17a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.362+0000 7f17c3fff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17a406ace0 0x7f17a406d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17c41f52d0 tx=0x7f17b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:01.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.366+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f17bc056ba0 con 0x7f17c40ceda0 2023-07-06T21:40:01.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.560+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f17c404e1d0 con 0x7f17c40ceda0 2023-07-06T21:40:01.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.561+0000 7f17c17fa700 1 -- 172.21.15.84:0/1190678214 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f17bc05a1c0 con 0x7f17c40ceda0 2023-07-06T21:40:01.564 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:01.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.563+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17a406ace0 msgr2=0x7f17a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.563+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17a406ace0 0x7f17a406d190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f17c41f52d0 tx=0x7f17b4008040 comp rx=0 tx=0).stop 2023-07-06T21:40:01.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.563+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 msgr2=0x7f17c40cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.563+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cb250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f17c406bed0 tx=0x7f17bc004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 shutdown_connections 2023-07-06T21:40:01.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f17a406ace0 0x7f17a406d190 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f17c40ceda0 0x7f17c40cb250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f17c40b1a80 0x7f17c41f1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 --2- 172.21.15.84:0/1190678214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17c40b81c0 0x7f17c40cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 >> 172.21.15.84:0/1190678214 conn(0x7f17c406b2b0 msgr2=0x7f17c40b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:01.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 shutdown_connections 2023-07-06T21:40:01.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.564+0000 7f17ca5ff700 1 -- 172.21.15.84:0/1190678214 wait complete. 2023-07-06T21:40:01.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:01.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.779+0000 7fc58c3ff700 1 Processor -- start 2023-07-06T21:40:01.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.780+0000 7fc58c3ff700 1 -- start start 2023-07-06T21:40:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 0x7fc5840d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58c3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5840d1830 con 0x7fc5840b81c0 2023-07-06T21:40:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58c3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5840cf330 con 0x7fc5840cfd10 2023-07-06T21:40:01.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5840d1270 con 0x7fc5840b1a80 2023-07-06T21:40:01.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58a19b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc5840b81c0 0x7fc5840cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56200/0 (socket says 172.21.15.84:56200) 2023-07-06T21:40:01.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58a19b700 1 -- 172.21.15.84:0/2826957985 learned_addr learned my addr 172.21.15.84:0/2826957985 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:01.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc58a99c700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 0x7fc5840d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.781+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654904542 0 0) 0x7fc5840d1830 con 0x7fc5840b81c0 2023-07-06T21:40:01.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5700035b0 con 0x7fc5840b81c0 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217613253 0 0) 0x7fc5840cf330 con 0x7fc5840cfd10 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5840d1830 con 0x7fc5840cfd10 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3963965316 0 0) 0x7fc5700035b0 con 0x7fc5840b81c0 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 msgr2=0x7fc5840d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 0x7fc5840d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 >> v1:172.21.15.107:6789/0 conn(0x7fc5840cfd10 legacy=0x7fc5840d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840d1550 con 0x7fc5840b81c0 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fc580001670 con 0x7fc5840b81c0 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 >> v1:172.21.15.84:6789/0 conn(0x7fc5840b81c0 legacy=0x7fc5840cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc570003bc0 0x7fc570003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570004850 0x7fc570004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 0x7fc570005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5840d1270 con 0x7fc570003bc0 2023-07-06T21:40:01.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57000bac0 con 0x7fc570004850 2023-07-06T21:40:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57000bc80 con 0x7fc5840b1a80 2023-07-06T21:40:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc58a19b700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc570003bc0 0x7fc570003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc58999a700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570004850 0x7fc570004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.782+0000 7fc58a99c700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 0x7fc570005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58a19b700 1 -- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 msgr2=0x7fc570005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:01.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58a19b700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5840b1a80 0x7fc570005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58a19b700 1 -- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570004850 msgr2=0x7fc570004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58a19b700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570004850 0x7fc570004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58a19b700 1 -- 172.21.15.84:0/2826957985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5700035b0 con 0x7fc570003bc0 2023-07-06T21:40:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58999a700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570004850 0x7fc570004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:01.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc58a19b700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc570003bc0 0x7fc570003fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc580001eb0 tx=0x7fc570005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25682a971de83e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000cc70 con 0x7fc570003bc0 2023-07-06T21:40:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58000cdf0 con 0x7fc570003bc0 2023-07-06T21:40:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.783+0000 7fc589199700 1 -- 172.21.15.84:0/2826957985 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580005ba0 con 0x7fc570003bc0 2023-07-06T21:40:01.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58c3ff700 1 -- 172.21.15.84:0/2826957985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc570003bc0 msgr2=0x7fc570003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/2826957985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc570003bc0 0x7fc570003fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc580001eb0 tx=0x7fc570005880 comp rx=0 tx=0).stop 2023-07-06T21:40:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58a99c700 1 -- 172.21.15.84:0/2826957985 reap_dead start 2023-07-06T21:40:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58c3ff700 1 -- 172.21.15.84:0/2826957985 shutdown_connections 2023-07-06T21:40:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58c3ff700 1 -- 172.21.15.84:0/2826957985 >> 172.21.15.84:0/2826957985 conn(0x7fc58406b2b0 msgr2=0x7fc58406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58c3ff700 1 -- 172.21.15.84:0/2826957985 shutdown_connections 2023-07-06T21:40:01.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.784+0000 7fc58c3ff700 1 -- 172.21.15.84:0/2826957985 wait complete. 2023-07-06T21:40:01.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.786+0000 7fc58c3ff700 1 Processor -- start 2023-07-06T21:40:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.786+0000 7fc58c3ff700 1 -- start start 2023-07-06T21:40:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.787+0000 7fc58c3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570003bc0 0x7fc584156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc584153aa0 0x7fc5841596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58c3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 0x7fc5841577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58c3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc584159bf0 con 0x7fc584157330 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58c3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc584157ce0 con 0x7fc570003bc0 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc584153f10 con 0x7fc584153aa0 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 0x7fc5841577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 0x7fc5841577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46894/0 (socket says 172.21.15.84:46894) 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 -- 172.21.15.84:0/3481178682 learned_addr learned my addr 172.21.15.84:0/3481178682 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a19b700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570003bc0 0x7fc584156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58999a700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc584153aa0 0x7fc5841596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 -- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc584153aa0 msgr2=0x7fc5841596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc584153aa0 0x7fc5841596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 -- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570003bc0 msgr2=0x7fc584156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570003bc0 0x7fc584156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.788+0000 7fc58a99c700 1 -- 172.21.15.84:0/3481178682 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5841f53f0 con 0x7fc584157330 2023-07-06T21:40:01.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.789+0000 7fc58a99c700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 0x7fc5841577a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5840c9c70 tx=0x7fc57c0114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.789+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00e070 con 0x7fc584157330 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.789+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c003cf0 con 0x7fc584157330 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.789+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c003ea0 con 0x7fc584157330 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.789+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5841f56d0 con 0x7fc584157330 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.789+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5841f5c90 con 0x7fc584157330 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.790+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc58404e1d0 con 0x7fc584157330 2023-07-06T21:40:01.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.791+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc57c011960 con 0x7fc584157330 2023-07-06T21:40:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.792+0000 7fc57affd700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc56806ace0 0x7fc56806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.792+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc57c089820 con 0x7fc584157330 2023-07-06T21:40:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.792+0000 7fc58a19b700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc56806ace0 0x7fc56806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:01.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.793+0000 7fc58a19b700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc56806ace0 0x7fc56806d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc580015040 tx=0x7fc580007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:01.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.796+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc57c0548c0 con 0x7fc584157330 2023-07-06T21:40:01.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.997+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5840b4f30 con 0x7fc584157330 2023-07-06T21:40:01.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.997+0000 7fc57affd700 1 -- 172.21.15.84:0/3481178682 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc57c057ee0 con 0x7fc584157330 2023-07-06T21:40:01.999 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:02.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc56806ace0 msgr2=0x7fc56806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc56806ace0 0x7fc56806d190 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc580015040 tx=0x7fc580007000 comp rx=0 tx=0).stop 2023-07-06T21:40:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 msgr2=0x7fc5841577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 0x7fc5841577a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5840c9c70 tx=0x7fc57c0114b0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 shutdown_connections 2023-07-06T21:40:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc56806ace0 0x7fc56806d190 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc584157330 0x7fc5841577a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc570003bc0 0x7fc584156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 --2- 172.21.15.84:0/3481178682 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc584153aa0 0x7fc5841596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 >> 172.21.15.84:0/3481178682 conn(0x7fc58406b2b0 msgr2=0x7fc584073410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 shutdown_connections 2023-07-06T21:40:02.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:01.999+0000 7fc58c3ff700 1 -- 172.21.15.84:0/3481178682 wait complete. 2023-07-06T21:40:02.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:02.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.216+0000 7f91e2fff700 1 Processor -- start 2023-07-06T21:40:02.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.216+0000 7f91e2fff700 1 -- start start 2023-07-06T21:40:02.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.217+0000 7f91e2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91dc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.217+0000 7f91e2fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f91dc0ceda0 0x7f91dc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.217+0000 7f91e2fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc0d1830 con 0x7f91dc0b1a80 2023-07-06T21:40:02.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.217+0000 7f91e2fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc0cf330 con 0x7f91dc0ceda0 2023-07-06T21:40:02.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.217+0000 7f91e2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0d1270 con 0x7f91dc0b81c0 2023-07-06T21:40:02.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91dbfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f91dc0b1a80 0x7f91dc15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56206/0 (socket says 172.21.15.84:56206) 2023-07-06T21:40:02.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91dbfff700 1 -- 172.21.15.84:0/422062309 learned_addr learned my addr 172.21.15.84:0/422062309 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 884154689 0 0) 0x7f91dc0d1830 con 0x7f91dc0b1a80 2023-07-06T21:40:02.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91dc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91e159c700 1 --2- 172.21.15.84:0/422062309 >> v2:172.21.15.107:3300/0 conn(0x7f91dc0ceda0 0x7f91dc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91cc0035b0 con 0x7f91dc0b1a80 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573305110 0 0) 0x7f91cc0035b0 con 0x7f91dc0b1a80 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 msgr2=0x7f91dc0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91dc0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 >> v2:172.21.15.107:3300/0 conn(0x7f91dc0ceda0 msgr2=0x7f91dc0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 --2- 172.21.15.84:0/422062309 >> v2:172.21.15.107:3300/0 conn(0x7f91dc0ceda0 0x7f91dc0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0d1550 con 0x7f91dc0b1a80 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f91c8001670 con 0x7f91dc0b1a80 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 >> v1:172.21.15.84:6789/0 conn(0x7f91dc0b1a80 legacy=0x7f91dc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91cc003a00 0x7f91cc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.218+0000 7f91db7fe700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc004850 0x7f91cc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91db7fe700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0cf330 con 0x7f91cc003a00 2023-07-06T21:40:02.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0d1270 con 0x7f91cc004850 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc00ba10 con 0x7f91dc0b81c0 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91cc003a00 0x7f91cc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e159c700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91dbfff700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc004850 0x7f91cc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 -- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 msgr2=0x7f91cc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91cc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 -- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc004850 msgr2=0x7f91cc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:02.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91cc004850 0x7f91cc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 -- 172.21.15.84:0/422062309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0035b0 con 0x7f91cc003a00 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e159c700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91dc0b81c0 0x7f91cc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91cc003a00 0x7f91cc003e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009b90 tx=0x7f91cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dc72ee608c0c2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.219+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000e6c0 con 0x7f91cc003a00 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d000e820 con 0x7f91cc003a00 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91db7fe700 1 -- 172.21.15.84:0/422062309 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d000eb90 con 0x7f91cc003a00 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e2fff700 1 -- 172.21.15.84:0/422062309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91cc003a00 msgr2=0x7f91cc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e2fff700 1 --2- 172.21.15.84:0/422062309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91cc003a00 0x7f91cc003e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009b90 tx=0x7f91cc005880 comp rx=0 tx=0).stop 2023-07-06T21:40:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e159c700 1 -- 172.21.15.84:0/422062309 reap_dead start 2023-07-06T21:40:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e2fff700 1 -- 172.21.15.84:0/422062309 shutdown_connections 2023-07-06T21:40:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e2fff700 1 -- 172.21.15.84:0/422062309 >> 172.21.15.84:0/422062309 conn(0x7f91dc06b2b0 msgr2=0x7f91dc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e2fff700 1 -- 172.21.15.84:0/422062309 shutdown_connections 2023-07-06T21:40:02.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.220+0000 7f91e2fff700 1 -- 172.21.15.84:0/422062309 wait complete. 2023-07-06T21:40:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.223+0000 7f91e2fff700 1 Processor -- start 2023-07-06T21:40:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.223+0000 7f91e2fff700 1 -- start start 2023-07-06T21:40:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91e2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91cc003a00 0x7f91dc1e2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91e2fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 0x7f91dc0b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91e2fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91dc1f4960 0x7f91dc1f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91e2fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0cc580 con 0x7f91dc0cbd00 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91e2fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0cc860 con 0x7f91dc1f4960 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91e2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0ccb40 con 0x7f91cc003a00 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91dbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 0x7f91dc0b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91dbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 0x7f91dc0b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46920/0 (socket says 172.21.15.84:46920) 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.224+0000 7f91dbfff700 1 -- 172.21.15.84:0/978731225 learned_addr learned my addr 172.21.15.84:0/978731225 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91e159c700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91dc1f4960 0x7f91dc1f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91cc003a00 0x7f91dc1e2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91dbfff700 1 -- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91cc003a00 msgr2=0x7f91dc1e2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91dbfff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91cc003a00 0x7f91dc1e2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91dbfff700 1 -- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91dc1f4960 msgr2=0x7f91dc1f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91dbfff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91dc1f4960 0x7f91dc1f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91dbfff700 1 -- 172.21.15.84:0/978731225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc0cce20 con 0x7f91dc0cbd00 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91dbfff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 0x7f91dc0b28c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e760 tx=0x7f91c800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8010510 con 0x7f91dc0cbd00 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8010670 con 0x7f91dc0cbd00 2023-07-06T21:40:02.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc1f52d0 con 0x7f91dc0cbd00 2023-07-06T21:40:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.225+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8010a20 con 0x7f91dc0cbd00 2023-07-06T21:40:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.226+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc0cc350 con 0x7f91dc0cbd00 2023-07-06T21:40:02.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.227+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc04e1d0 con 0x7f91dc0cbd00 2023-07-06T21:40:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.227+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91c8010b80 con 0x7f91dc0cbd00 2023-07-06T21:40:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.228+0000 7f91d97fa700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91c006ace0 0x7f91c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.229+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f91c8088cf0 con 0x7f91dc0cbd00 2023-07-06T21:40:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.229+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91c006ace0 0x7f91c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.229+0000 7f91e0d9b700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91c006ace0 0x7f91c006d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f91d00090b0 tx=0x7f91d001f040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:02.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.233+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91c8053d90 con 0x7f91dc0cbd00 2023-07-06T21:40:02.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.427+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91dc0b4f30 con 0x7f91dc0cbd00 2023-07-06T21:40:02.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.428+0000 7f91d97fa700 1 -- 172.21.15.84:0/978731225 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91c8019020 con 0x7f91dc0cbd00 2023-07-06T21:40:02.430 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:02.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91c006ace0 msgr2=0x7f91c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91c006ace0 0x7f91c006d190 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f91d00090b0 tx=0x7f91d001f040 comp rx=0 tx=0).stop 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 msgr2=0x7f91dc0b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 0x7f91dc0b28c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f91c800e760 tx=0x7f91c800ea70 comp rx=0 tx=0).stop 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 shutdown_connections 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91c006ace0 0x7f91c006d190 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91dc1f4960 0x7f91dc1f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91cc003a00 0x7f91dc1e2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 --2- 172.21.15.84:0/978731225 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91dc0cbd00 0x7f91dc0b28c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 >> 172.21.15.84:0/978731225 conn(0x7f91dc06b2b0 msgr2=0x7f91dc0b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.430+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 shutdown_connections 2023-07-06T21:40:02.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.431+0000 7f91e2fff700 1 -- 172.21.15.84:0/978731225 wait complete. 2023-07-06T21:40:02.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:02.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.642+0000 7fe3c1dff700 1 Processor -- start 2023-07-06T21:40:02.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.642+0000 7fe3c1dff700 1 -- start start 2023-07-06T21:40:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3c1dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3bc0b83c0 0x7fe3bc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3c1dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3bc0d3710 0x7fe3bc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3c1dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc0d0a80 con 0x7fe3bc0cfa90 2023-07-06T21:40:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3c1dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cff10 con 0x7fe3bc0d3710 2023-07-06T21:40:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3c1dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0b8790 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3bc0d3710 0x7fe3bc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe3bc0d3710 0x7fe3bc0cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46934/0 (socket says 172.21.15.84:46934) 2023-07-06T21:40:02.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 -- 172.21.15.84:0/3777690518 learned_addr learned my addr 172.21.15.84:0/3777690518 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:02.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 -- 172.21.15.84:0/3777690518 >> v2:172.21.15.107:3300/0 conn(0x7fe3bc0b83c0 msgr2=0x7fe3bc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:02.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3bb7fe700 1 --2- 172.21.15.84:0/3777690518 >> v2:172.21.15.107:3300/0 conn(0x7fe3bc0b83c0 0x7fe3bc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 --2- 172.21.15.84:0/3777690518 >> v2:172.21.15.107:3300/0 conn(0x7fe3bc0b83c0 0x7fe3bc0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 -- 172.21.15.84:0/3777690518 >> v1:172.21.15.107:6789/0 conn(0x7fe3bc0cfa90 legacy=0x7fe3bc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3b3fff700 1 -- 172.21.15.84:0/3777690518 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc0b4e70 con 0x7fe3bc0d3710 2023-07-06T21:40:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.644+0000 7fe3bb7fe700 1 --2- 172.21.15.84:0/3777690518 >> v2:172.21.15.107:3300/0 conn(0x7fe3bc0b83c0 0x7fe3bc0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.645+0000 7fe3b3fff700 1 --2- 172.21.15.84:0/3777690518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0d3710 0x7fe3bc0cf550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0cf520 tx=0x7fe3a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e416a4cd876d53 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:02.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.645+0000 7fe3baffd700 1 -- 172.21.15.84:0/3777690518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4004230 con 0x7fe3bc0d3710 2023-07-06T21:40:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.645+0000 7fe3baffd700 1 -- 172.21.15.84:0/3777690518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4004390 con 0x7fe3bc0d3710 2023-07-06T21:40:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.645+0000 7fe3baffd700 1 -- 172.21.15.84:0/3777690518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a40036a0 con 0x7fe3bc0d3710 2023-07-06T21:40:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.645+0000 7fe3c1dff700 1 -- 172.21.15.84:0/3777690518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0d3710 msgr2=0x7fe3bc0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.645+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/3777690518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0d3710 0x7fe3bc0cf550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0cf520 tx=0x7fe3a4009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.646+0000 7fe3c1dff700 1 -- 172.21.15.84:0/3777690518 shutdown_connections 2023-07-06T21:40:02.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.646+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/3777690518 >> v2:172.21.15.107:3300/0 conn(0x7fe3bc0b83c0 0x7fe3bc0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.646+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/3777690518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0d3710 0x7fe3bc0cf550 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.646+0000 7fe3c1dff700 1 -- 172.21.15.84:0/3777690518 >> 172.21.15.84:0/3777690518 conn(0x7fe3bc06b350 msgr2=0x7fe3bc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.646+0000 7fe3c1dff700 1 -- 172.21.15.84:0/3777690518 shutdown_connections 2023-07-06T21:40:02.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.646+0000 7fe3c1dff700 1 -- 172.21.15.84:0/3777690518 wait complete. 2023-07-06T21:40:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.647+0000 7fe3c1dff700 1 Processor -- start 2023-07-06T21:40:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.647+0000 7fe3c1dff700 1 -- start start 2023-07-06T21:40:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3c1dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 0x7fe3bc1f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3c1dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3bc0cfa90 0x7fe3bc0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3c1dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3bc0d3710 0x7fe3bc0b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3c1dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc15ea90 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3c1dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc1f64a0 con 0x7fe3bc0cfa90 2023-07-06T21:40:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3c1dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc15f5d0 con 0x7fe3bc0d3710 2023-07-06T21:40:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.648+0000 7fe3bb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 0x7fe3bc1f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 0x7fe3bc1f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46950/0 (socket says 172.21.15.84:46950) 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 -- 172.21.15.84:0/4260802294 learned_addr learned my addr 172.21.15.84:0/4260802294 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3b3fff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3bc0cfa90 0x7fe3bc0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bbfff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3bc0d3710 0x7fe3bc0b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 -- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3bc0d3710 msgr2=0x7fe3bc0b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3bc0d3710 0x7fe3bc0b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 -- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3bc0cfa90 msgr2=0x7fe3bc0b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3bc0cfa90 0x7fe3bc0b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 -- 172.21.15.84:0/4260802294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4009550 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bbfff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3bc0d3710 0x7fe3bc0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3bb7fe700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 0x7fe3bc1f5f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c7f0 tx=0x7fe3ac00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac011800 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac011960 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac011cf0 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.649+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc1f2070 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.650+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc1f2630 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.650+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc0b2a70 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.651+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3ac018360 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.652+0000 7fe3b8ff9700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe39c06ace0 0x7fe39c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:02.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.652+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3ac089540 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.653+0000 7fe3b3fff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe39c06ace0 0x7fe39c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:02.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.653+0000 7fe3b3fff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe39c06ace0 0x7fe39c06d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0d3b20 tx=0x7fe3a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:02.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.657+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3ac054560 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.855+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3bc0b2790 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.856+0000 7fe3b8ff9700 1 -- 172.21.15.84:0/4260802294 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7fe3ac057b80 con 0x7fe3bc0b83c0 2023-07-06T21:40:02.858 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:02.859 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:02.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe39c06ace0 msgr2=0x7fe39c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe39c06ace0 0x7fe39c06d190 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc0d3b20 tx=0x7fe3a4008030 comp rx=0 tx=0).stop 2023-07-06T21:40:02.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 msgr2=0x7fe3bc1f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:02.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 0x7fe3bc1f5f60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c7f0 tx=0x7fe3ac00cb00 comp rx=0 tx=0).stop 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 shutdown_connections 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe39c06ace0 0x7fe39c06d190 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3bc0d3710 0x7fe3bc0b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe3bc0b83c0 0x7fe3bc1f5f60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 --2- 172.21.15.84:0/4260802294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3bc0cfa90 0x7fe3bc0b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 >> 172.21.15.84:0/4260802294 conn(0x7fe3bc06b350 msgr2=0x7fe3bc0b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 shutdown_connections 2023-07-06T21:40:02.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:02.859+0000 7fe3c1dff700 1 -- 172.21.15.84:0/4260802294 wait complete. 2023-07-06T21:40:02.879 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:02.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:03.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.076+0000 7f80a8bff700 1 Processor -- start 2023-07-06T21:40:03.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.076+0000 7f80a8bff700 1 -- start start 2023-07-06T21:40:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.077+0000 7f80a8bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f80a4071630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a8bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a406a8f0 con 0x7f80a406b5f0 2023-07-06T21:40:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a8bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a4071d80 con 0x7f80a406cf30 2023-07-06T21:40:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a8bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40cb9c0 con 0x7f80a406b220 2023-07-06T21:40:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f809bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f80a406b5f0 0x7f80a40cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55902/0 (socket says 172.21.15.84:55902) 2023-07-06T21:40:03.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f809bfff700 1 -- 172.21.15.84:0/4218867376 learned_addr learned my addr 172.21.15.84:0/4218867376 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:03.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a259c700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f80a4071630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2205995010 0 0) 0x7f80a406a8f0 con 0x7f80a406b5f0 2023-07-06T21:40:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80900035b0 con 0x7f80a406b5f0 2023-07-06T21:40:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347696980 0 0) 0x7f80a4071d80 con 0x7f80a406cf30 2023-07-06T21:40:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a406a8f0 con 0x7f80a406cf30 2023-07-06T21:40:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 862788765 0 0) 0x7f80900035b0 con 0x7f80a406b5f0 2023-07-06T21:40:03.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 msgr2=0x7f80a4071630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f80a4071630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 >> v1:172.21.15.107:6789/0 conn(0x7f80a406cf30 legacy=0x7f80a4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.078+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40ca7d0 con 0x7f80a406b5f0 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f808c001670 con 0x7f80a406b5f0 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 >> v1:172.21.15.84:6789/0 conn(0x7f80a406b5f0 legacy=0x7f80a40cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f8090003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f8090004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8090004d90 0x7f8090005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40cb9c0 con 0x7f8090003bc0 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809000bac0 con 0x7f8090004d90 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809000bc80 con 0x7f80a406b220 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a259c700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f8090003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f809bfff700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f8090004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a2d9d700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8090004d90 0x7f8090005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a259c700 1 -- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 msgr2=0x7f8090004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a259c700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f8090004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a259c700 1 -- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8090004d90 msgr2=0x7f8090005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a259c700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8090004d90 0x7f8090005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f80a259c700 1 -- 172.21.15.84:0/4218867376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80900035b0 con 0x7f8090003bc0 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.079+0000 7f809bfff700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a406b220 0x7f8090004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a259c700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f8090003fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8094009b30 tx=0x7f8090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fab445ccb7dbc05 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400e6c0 con 0x7f8090003bc0 2023-07-06T21:40:03.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809400e820 con 0x7f8090003bc0 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a1d9b700 1 -- 172.21.15.84:0/4218867376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400eb90 con 0x7f8090003bc0 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a8bff700 1 -- 172.21.15.84:0/4218867376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 msgr2=0x7f8090003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a8bff700 1 --2- 172.21.15.84:0/4218867376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f8090003fb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8094009b30 tx=0x7f8090005880 comp rx=0 tx=0).stop 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a2d9d700 1 -- 172.21.15.84:0/4218867376 reap_dead start 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a8bff700 1 -- 172.21.15.84:0/4218867376 shutdown_connections 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a8bff700 1 -- 172.21.15.84:0/4218867376 >> 172.21.15.84:0/4218867376 conn(0x7f80a40cf080 msgr2=0x7f80a40cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a8bff700 1 -- 172.21.15.84:0/4218867376 shutdown_connections 2023-07-06T21:40:03.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.080+0000 7f80a8bff700 1 -- 172.21.15.84:0/4218867376 wait complete. 2023-07-06T21:40:03.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.082+0000 7f80a8bff700 1 Processor -- start 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.082+0000 7f80a8bff700 1 -- start start 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.083+0000 7f80a8bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f80a40c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.083+0000 7f80a8bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a40c28c0 0x7f80a40c2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.083+0000 7f80a8bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a414eec0 0x7f80a414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.083+0000 7f80a8bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40c5f50 con 0x7f8090003bc0 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.083+0000 7f80a8bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a415a270 con 0x7f80a40c28c0 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.083+0000 7f80a8bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a414f870 con 0x7f80a414eec0 2023-07-06T21:40:03.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f80a40c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f80a40c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57062/0 (socket says 172.21.15.84:57062) 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 -- 172.21.15.84:0/1875463024 learned_addr learned my addr 172.21.15.84:0/1875463024 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f809bfff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a40c28c0 0x7f80a40c2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a2d9d700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a414eec0 0x7f80a414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 -- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a414eec0 msgr2=0x7f80a414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a414eec0 0x7f80a414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 -- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a40c28c0 msgr2=0x7f80a40c2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a40c28c0 0x7f80a40c2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 -- 172.21.15.84:0/1875463024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a40c50f0 con 0x7f8090003bc0 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f809bfff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a40c28c0 0x7f80a40c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f80a259c700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f80a40c8f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8094004270 tx=0x7f80940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.084+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8094021070 con 0x7f8090003bc0 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.085+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80940036a0 con 0x7f8090003bc0 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.085+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809400e6c0 con 0x7f8090003bc0 2023-07-06T21:40:03.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.085+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a40c9470 con 0x7f8090003bc0 2023-07-06T21:40:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.085+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a40c5620 con 0x7f8090003bc0 2023-07-06T21:40:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.086+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a409d4a0 con 0x7f8090003bc0 2023-07-06T21:40:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.087+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8094025070 con 0x7f8090003bc0 2023-07-06T21:40:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.087+0000 7f809affd700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f808406ace0 0x7f808406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.088+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f809408a9d0 con 0x7f8090003bc0 2023-07-06T21:40:03.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.088+0000 7f809bfff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f808406ace0 0x7f808406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.088+0000 7f809bfff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f808406ace0 0x7f808406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80a40cce90 tx=0x7f808c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.092+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8094055b90 con 0x7f8090003bc0 2023-07-06T21:40:03.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.287+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80a404e1d0 con 0x7f8090003bc0 2023-07-06T21:40:03.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.288+0000 7f809affd700 1 -- 172.21.15.84:0/1875463024 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80940591b0 con 0x7f8090003bc0 2023-07-06T21:40:03.291 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:03.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f808406ace0 msgr2=0x7f808406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f808406ace0 0x7f808406d190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f80a40cce90 tx=0x7f808c009040 comp rx=0 tx=0).stop 2023-07-06T21:40:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 msgr2=0x7f80a40c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f80a40c8f30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8094004270 tx=0x7f80940047c0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 shutdown_connections 2023-07-06T21:40:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f808406ace0 0x7f808406d190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80a414eec0 0x7f80a414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8090003bc0 0x7f80a40c8f30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 --2- 172.21.15.84:0/1875463024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80a40c28c0 0x7f80a40c2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 >> 172.21.15.84:0/1875463024 conn(0x7f80a40cf080 msgr2=0x7f80a40cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:03.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 shutdown_connections 2023-07-06T21:40:03.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.290+0000 7f80a8bff700 1 -- 172.21.15.84:0/1875463024 wait complete. 2023-07-06T21:40:03.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:03.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.504+0000 7f63641ff700 1 Processor -- start 2023-07-06T21:40:03.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.504+0000 7f63641ff700 1 -- start start 2023-07-06T21:40:03.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f63641ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b1a80 0x7f635c0d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f63641ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c0d1830 con 0x7f635c0b81c0 2023-07-06T21:40:03.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f63641ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f635c0cf330 con 0x7f635c0cfd10 2023-07-06T21:40:03.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f63641ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c0d1270 con 0x7f635c0b1a80 2023-07-06T21:40:03.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f6361f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f635c0b81c0 0x7f635c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55918/0 (socket says 172.21.15.84:55918) 2023-07-06T21:40:03.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f6361f9b700 1 -- 172.21.15.84:0/3353561283 learned_addr learned my addr 172.21.15.84:0/3353561283 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:03.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f636279c700 1 --2- 172.21.15.84:0/3353561283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b1a80 0x7f635c0d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875581235 0 0) 0x7f635c0d1830 con 0x7f635c0b81c0 2023-07-06T21:40:03.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63480035b0 con 0x7f635c0b81c0 2023-07-06T21:40:03.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.506+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510376542 0 0) 0x7f635c0cf330 con 0x7f635c0cfd10 2023-07-06T21:40:03.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f635c0d1830 con 0x7f635c0cfd10 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1588614225 0 0) 0x7f63480035b0 con 0x7f635c0b81c0 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f636279c700 1 -- 172.21.15.84:0/3353561283 >> v1:172.21.15.107:6789/0 conn(0x7f635c0cfd10 legacy=0x7f635c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f636279c700 1 -- 172.21.15.84:0/3353561283 >> v1:172.21.15.84:6789/0 conn(0x7f635c0b81c0 legacy=0x7f635c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f636279c700 1 -- 172.21.15.84:0/3353561283 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c0d1550 con 0x7f635c0b1a80 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f636279c700 1 --2- 172.21.15.84:0/3353561283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b1a80 0x7f635c0d2a90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6354009dc0 tx=0x7f635400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d85c6153d1a33a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635400e070 con 0x7f635c0b1a80 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6354003a20 con 0x7f635c0b1a80 2023-07-06T21:40:03.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.507+0000 7f6360f99700 1 -- 172.21.15.84:0/3353561283 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358001670 con 0x7f635c0b1a80 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 -- 172.21.15.84:0/3353561283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b1a80 msgr2=0x7f635c0d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 --2- 172.21.15.84:0/3353561283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b1a80 0x7f635c0d2a90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6354009dc0 tx=0x7f635400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 -- 172.21.15.84:0/3353561283 shutdown_connections 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 --2- 172.21.15.84:0/3353561283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b1a80 0x7f635c0d2a90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 -- 172.21.15.84:0/3353561283 >> 172.21.15.84:0/3353561283 conn(0x7f635c06b2b0 msgr2=0x7f635c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 -- 172.21.15.84:0/3353561283 shutdown_connections 2023-07-06T21:40:03.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.508+0000 7f63641ff700 1 -- 172.21.15.84:0/3353561283 wait complete. 2023-07-06T21:40:03.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.510+0000 7f63641ff700 1 Processor -- start 2023-07-06T21:40:03.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.510+0000 7f63641ff700 1 -- start start 2023-07-06T21:40:03.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f63641ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 0x7f635c1f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f63641ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b81c0 0x7f635c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f63641ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f635c0cfd10 0x7f635c150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f63641ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c150da0 con 0x7f635c0b1a80 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f63641ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c1f6980 con 0x7f635c0cfd10 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f63641ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c1f60e0 con 0x7f635c0b81c0 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 0x7f635c1f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 0x7f635c1f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57076/0 (socket says 172.21.15.84:57076) 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 -- 172.21.15.84:0/807196588 learned_addr learned my addr 172.21.15.84:0/807196588 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f636279c700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f635c0cfd10 0x7f635c150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f636179a700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b81c0 0x7f635c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 -- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b81c0 msgr2=0x7f635c0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b81c0 0x7f635c0c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 -- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f635c0cfd10 msgr2=0x7f635c150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f635c0cfd10 0x7f635c150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f6361f9b700 1 -- 172.21.15.84:0/807196588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6354009a70 con 0x7f635c0b1a80 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.511+0000 7f636279c700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f635c0cfd10 0x7f635c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.512+0000 7f6361f9b700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 0x7f635c1f59e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f635800c750 tx=0x7f635800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.512+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358004c80 con 0x7f635c0b1a80 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.512+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6358004e00 con 0x7f635c0b1a80 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.512+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635c1f63c0 con 0x7f635c0b1a80 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.512+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358007310 con 0x7f635c0b1a80 2023-07-06T21:40:03.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.512+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635c0c8500 con 0x7f635c0b1a80 2023-07-06T21:40:03.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.513+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635c0c8360 con 0x7f635c0b1a80 2023-07-06T21:40:03.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.514+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6358007470 con 0x7f635c0b1a80 2023-07-06T21:40:03.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.515+0000 7f63527fc700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f634006acb0 0x7f634006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.515+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6358089280 con 0x7f635c0b1a80 2023-07-06T21:40:03.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.515+0000 7f636179a700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f634006acb0 0x7f634006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.516+0000 7f636179a700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f634006acb0 0x7f634006d160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f634c0011e0 tx=0x7f634c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.519+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f635c0c8360 con 0x7f635c0b1a80 2023-07-06T21:40:03.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.716+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f635c0c8360 con 0x7f635c0b1a80 2023-07-06T21:40:03.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.717+0000 7f63527fc700 1 -- 172.21.15.84:0/807196588 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f635c0c8360 con 0x7f635c0b1a80 2023-07-06T21:40:03.719 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f634006acb0 msgr2=0x7f634006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f634006acb0 0x7f634006d160 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f634c0011e0 tx=0x7f634c009040 comp rx=0 tx=0).stop 2023-07-06T21:40:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 msgr2=0x7f635c1f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 0x7f635c1f59e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f635800c750 tx=0x7f635800ca60 comp rx=0 tx=0).stop 2023-07-06T21:40:03.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 shutdown_connections 2023-07-06T21:40:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f634006acb0 0x7f634006d160 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f635c0cfd10 0x7f635c150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f635c0b1a80 0x7f635c1f59e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 --2- 172.21.15.84:0/807196588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f635c0b81c0 0x7f635c0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 >> 172.21.15.84:0/807196588 conn(0x7f635c06b2b0 msgr2=0x7f635c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:03.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 shutdown_connections 2023-07-06T21:40:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.719+0000 7f63641ff700 1 -- 172.21.15.84:0/807196588 wait complete. 2023-07-06T21:40:03.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:03.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.932+0000 7f1e79fff700 1 Processor -- start 2023-07-06T21:40:03.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.932+0000 7f1e79fff700 1 -- start start 2023-07-06T21:40:03.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.933+0000 7f1e79fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1e7415c750 0x7f1e740ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.933+0000 7f1e79fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e740ad960 0x7f1e740add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.933+0000 7f1e79fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e740cf690 con 0x7f1e740ac450 2023-07-06T21:40:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.933+0000 7f1e79fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740ae2a0 con 0x7f1e740ad960 2023-07-06T21:40:03.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e79fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e740ac8c0 con 0x7f1e7415c750 2023-07-06T21:40:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e740ad960 0x7f1e740add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1e740ad960 0x7f1e740add60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57084/0 (socket says 172.21.15.84:57084) 2023-07-06T21:40:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 -- 172.21.15.84:0/1317212989 learned_addr learned my addr 172.21.15.84:0/1317212989 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:03.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e737fe700 1 --2- 172.21.15.84:0/1317212989 >> v2:172.21.15.107:3300/0 conn(0x7f1e7415c750 0x7f1e740ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 -- 172.21.15.84:0/1317212989 >> v2:172.21.15.107:3300/0 conn(0x7f1e7415c750 msgr2=0x7f1e740ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 --2- 172.21.15.84:0/1317212989 >> v2:172.21.15.107:3300/0 conn(0x7f1e7415c750 0x7f1e740ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 -- 172.21.15.84:0/1317212989 >> v1:172.21.15.107:6789/0 conn(0x7f1e740ac450 legacy=0x7f1e740c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 -- 172.21.15.84:0/1317212989 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740acba0 con 0x7f1e740ad960 2023-07-06T21:40:03.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.934+0000 7f1e73fff700 1 --2- 172.21.15.84:0/1317212989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e740add60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e740726a0 tx=0x7f1e6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb7762e8483abad server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e727fc700 1 -- 172.21.15.84:0/1317212989 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00e070 con 0x7f1e740ad960 2023-07-06T21:40:03.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e727fc700 1 -- 172.21.15.84:0/1317212989 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6c00b9c0 con 0x7f1e740ad960 2023-07-06T21:40:03.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e727fc700 1 -- 172.21.15.84:0/1317212989 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6c00bd70 con 0x7f1e740ad960 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 -- 172.21.15.84:0/1317212989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 msgr2=0x7f1e740add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 --2- 172.21.15.84:0/1317212989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e740add60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1e740726a0 tx=0x7f1e6c00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 -- 172.21.15.84:0/1317212989 shutdown_connections 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 --2- 172.21.15.84:0/1317212989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e740add60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 --2- 172.21.15.84:0/1317212989 >> v2:172.21.15.107:3300/0 conn(0x7f1e7415c750 0x7f1e740ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 -- 172.21.15.84:0/1317212989 >> 172.21.15.84:0/1317212989 conn(0x7f1e7406b8c0 msgr2=0x7f1e740cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 -- 172.21.15.84:0/1317212989 shutdown_connections 2023-07-06T21:40:03.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.935+0000 7f1e79fff700 1 -- 172.21.15.84:0/1317212989 wait complete. 2023-07-06T21:40:03.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.937+0000 7f1e79fff700 1 Processor -- start 2023-07-06T21:40:03.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.937+0000 7f1e79fff700 1 -- start start 2023-07-06T21:40:03.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.938+0000 7f1e79fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e740ac450 0x7f1e740c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.938+0000 7f1e79fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e7415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.938+0000 7f1e79fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e7415c750 0x7f1e7415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.938+0000 7f1e79fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c4d80 con 0x7f1e740ad960 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.938+0000 7f1e79fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c4a50 con 0x7f1e740ac450 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.938+0000 7f1e79fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e740c9680 con 0x7f1e7415c750 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e7415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e7415aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57098/0 (socket says 172.21.15.84:57098) 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 -- 172.21.15.84:0/3047699829 learned_addr learned my addr 172.21.15.84:0/3047699829 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:03.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e737fe700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e740ac450 0x7f1e740c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e73fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e7415c750 0x7f1e7415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 -- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e7415c750 msgr2=0x7f1e7415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e7415c750 0x7f1e7415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 -- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e740ac450 msgr2=0x7f1e740c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e740ac450 0x7f1e740c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 -- 172.21.15.84:0/3047699829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c009b30 con 0x7f1e740ad960 2023-07-06T21:40:03.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e73fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e7415c750 0x7f1e7415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.939+0000 7f1e72ffd700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e7415aa60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800c4e0 tx=0x7f1e6800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.940+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680052d0 con 0x7f1e740ad960 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.940+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e68005430 con 0x7f1e740ad960 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.940+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e680038c0 con 0x7f1e740ad960 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.940+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e740c9960 con 0x7f1e740ad960 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.940+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7415b1e0 con 0x7f1e740ad960 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.941+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e740c9b50 con 0x7f1e740ad960 2023-07-06T21:40:03.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.942+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e68003a20 con 0x7f1e740ad960 2023-07-06T21:40:03.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.943+0000 7f1e5bfff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e6006ace0 0x7f1e6006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:03.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.943+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1e68088900 con 0x7f1e740ad960 2023-07-06T21:40:03.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.944+0000 7f1e737fe700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e6006ace0 0x7f1e6006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:03.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.944+0000 7f1e737fe700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e6006ace0 0x7f1e6006d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006fd0 tx=0x7f1e64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:03.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:03.948+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e68053910 con 0x7f1e740ad960 2023-07-06T21:40:04.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.144+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e7404e1d0 con 0x7f1e740ad960 2023-07-06T21:40:04.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.145+0000 7f1e5bfff700 1 -- 172.21.15.84:0/3047699829 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e68056f30 con 0x7f1e740ad960 2023-07-06T21:40:04.147 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:04.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e6006ace0 msgr2=0x7f1e6006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e6006ace0 0x7f1e6006d190 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006fd0 tx=0x7f1e64008040 comp rx=0 tx=0).stop 2023-07-06T21:40:04.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 msgr2=0x7f1e7415aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e7415aa60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800c4e0 tx=0x7f1e6800c7f0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 shutdown_connections 2023-07-06T21:40:04.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e6006ace0 0x7f1e6006d190 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e7415c750 0x7f1e7415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e740ac450 0x7f1e740c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 --2- 172.21.15.84:0/3047699829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e740ad960 0x7f1e7415aa60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 >> 172.21.15.84:0/3047699829 conn(0x7f1e7406b8c0 msgr2=0x7f1e7406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 shutdown_connections 2023-07-06T21:40:04.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.147+0000 7f1e79fff700 1 -- 172.21.15.84:0/3047699829 wait complete. 2023-07-06T21:40:04.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.362+0000 7fd88e9ff700 1 Processor -- start 2023-07-06T21:40:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.363+0000 7fd88e9ff700 1 -- start start 2023-07-06T21:40:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd88e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880cfd10 0x7fd8880d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd88e9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880d1830 con 0x7fd8880b1a80 2023-07-06T21:40:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd88e9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8880cf330 con 0x7fd8880b81c0 2023-07-06T21:40:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd88e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8880d1270 con 0x7fd8880cfd10 2023-07-06T21:40:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd88cf9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd8880b1a80 0x7fd8880d28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55924/0 (socket says 172.21.15.84:55924) 2023-07-06T21:40:04.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd88cf9c700 1 -- 172.21.15.84:0/2034745527 learned_addr learned my addr 172.21.15.84:0/2034745527 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.364+0000 7fd8877fe700 1 --2- 172.21.15.84:0/2034745527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880cfd10 0x7fd8880d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442441461 0 0) 0x7fd8880d1830 con 0x7fd8880b1a80 2023-07-06T21:40:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8700035b0 con 0x7fd8880b1a80 2023-07-06T21:40:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591513905 0 0) 0x7fd8880cf330 con 0x7fd8880b81c0 2023-07-06T21:40:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8880d1830 con 0x7fd8880b81c0 2023-07-06T21:40:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd8877fe700 1 -- 172.21.15.84:0/2034745527 >> v1:172.21.15.107:6789/0 conn(0x7fd8880b81c0 legacy=0x7fd8880cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd8877fe700 1 -- 172.21.15.84:0/2034745527 >> v1:172.21.15.84:6789/0 conn(0x7fd8880b1a80 legacy=0x7fd8880d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd8877fe700 1 -- 172.21.15.84:0/2034745527 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880d1550 con 0x7fd8880cfd10 2023-07-06T21:40:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd8877fe700 1 --2- 172.21.15.84:0/2034745527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880cfd10 0x7fd8880d2380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098a0 tx=0x7fd87c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df89ecf232220b62 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011830 con 0x7fd8880cfd10 2023-07-06T21:40:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c011990 con 0x7fd8880cfd10 2023-07-06T21:40:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.365+0000 7fd886ffd700 1 -- 172.21.15.84:0/2034745527 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011d00 con 0x7fd8880cfd10 2023-07-06T21:40:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2034745527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880cfd10 msgr2=0x7fd8880d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2034745527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880cfd10 0x7fd8880d2380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0098a0 tx=0x7fd87c009bb0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2034745527 shutdown_connections 2023-07-06T21:40:04.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2034745527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880cfd10 0x7fd8880d2380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2034745527 >> 172.21.15.84:0/2034745527 conn(0x7fd88806b2b0 msgr2=0x7fd88806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:04.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2034745527 shutdown_connections 2023-07-06T21:40:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.366+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2034745527 wait complete. 2023-07-06T21:40:04.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.368+0000 7fd88e9ff700 1 Processor -- start 2023-07-06T21:40:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.368+0000 7fd88e9ff700 1 -- start start 2023-07-06T21:40:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880b1a80 0x7fd88815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88e9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8880b81c0 0x7fd888148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88e9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 0x7fd8881f2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88e9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88815d970 con 0x7fd8880cfd10 2023-07-06T21:40:04.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88e9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88815e1f0 con 0x7fd8880b81c0 2023-07-06T21:40:04.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88815e4d0 con 0x7fd8880b1a80 2023-07-06T21:40:04.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88cf9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 0x7fd8881f2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88cf9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 0x7fd8881f2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57110/0 (socket says 172.21.15.84:57110) 2023-07-06T21:40:04.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd88cf9c700 1 -- 172.21.15.84:0/2152910430 learned_addr learned my addr 172.21.15.84:0/2152910430 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:04.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd8877fe700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8880b81c0 0x7fd888148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.369+0000 7fd887fff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880b1a80 0x7fd88815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88cf9c700 1 -- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880b1a80 msgr2=0x7fd88815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88cf9c700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880b1a80 0x7fd88815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88cf9c700 1 -- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8880b81c0 msgr2=0x7fd888148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88cf9c700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8880b81c0 0x7fd888148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88cf9c700 1 -- 172.21.15.84:0/2152910430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c0094a0 con 0x7fd8880cfd10 2023-07-06T21:40:04.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd887fff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880b1a80 0x7fd88815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:04.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88cf9c700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 0x7fd8881f2060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd88000cba0 tx=0x7fd88000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:04.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000aa10 con 0x7fd8880cfd10 2023-07-06T21:40:04.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000ab70 con 0x7fd8880cfd10 2023-07-06T21:40:04.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88815eb60 con 0x7fd8880cfd10 2023-07-06T21:40:04.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.370+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880012400 con 0x7fd8880cfd10 2023-07-06T21:40:04.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.371+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd888149370 con 0x7fd8880cfd10 2023-07-06T21:40:04.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.371+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88815f080 con 0x7fd8880cfd10 2023-07-06T21:40:04.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.372+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd88001b3b0 con 0x7fd8880cfd10 2023-07-06T21:40:04.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.373+0000 7fd884ff9700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd87406ace0 0x7fd87406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.373+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd880089280 con 0x7fd8880cfd10 2023-07-06T21:40:04.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.374+0000 7fd887fff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd87406ace0 0x7fd87406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.374+0000 7fd887fff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd87406ace0 0x7fd87406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8780011e0 tx=0x7fd878009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:04.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.377+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd88815f080 con 0x7fd8880cfd10 2023-07-06T21:40:04.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.571+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd88815f080 con 0x7fd8880cfd10 2023-07-06T21:40:04.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.572+0000 7fd884ff9700 1 -- 172.21.15.84:0/2152910430 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd88005d020 con 0x7fd8880cfd10 2023-07-06T21:40:04.574 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd87406ace0 msgr2=0x7fd87406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd87406ace0 0x7fd87406d190 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd8780011e0 tx=0x7fd878009040 comp rx=0 tx=0).stop 2023-07-06T21:40:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 msgr2=0x7fd8881f2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 0x7fd8881f2060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd88000cba0 tx=0x7fd88000ceb0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 shutdown_connections 2023-07-06T21:40:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd87406ace0 0x7fd87406d190 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8880cfd10 0x7fd8881f2060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8880b1a80 0x7fd88815d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 --2- 172.21.15.84:0/2152910430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8880b81c0 0x7fd888148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 >> 172.21.15.84:0/2152910430 conn(0x7fd88806b2b0 msgr2=0x7fd8880cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 shutdown_connections 2023-07-06T21:40:04.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.574+0000 7fd88e9ff700 1 -- 172.21.15.84:0/2152910430 wait complete. 2023-07-06T21:40:04.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:04.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.785+0000 7eff3f5ff700 1 Processor -- start 2023-07-06T21:40:04.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.785+0000 7eff3f5ff700 1 -- start start 2023-07-06T21:40:04.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.786+0000 7eff3f5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff380cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.786+0000 7eff3f5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7eff380d3710 0x7eff380cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.786+0000 7eff3f5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff380cfee0 con 0x7eff380cfa90 2023-07-06T21:40:04.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.786+0000 7eff3f5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff380d0a80 con 0x7eff380d3710 2023-07-06T21:40:04.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.786+0000 7eff3f5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff380b8790 con 0x7eff380b83c0 2023-07-06T21:40:04.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3d39b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff380cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3d39b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff380cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:34394/0 (socket says 172.21.15.84:34394) 2023-07-06T21:40:04.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3d39b700 1 -- 172.21.15.84:0/1099118819 learned_addr learned my addr 172.21.15.84:0/1099118819 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:04.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3cb9a700 1 --2- 172.21.15.84:0/1099118819 >> v2:172.21.15.107:3300/0 conn(0x7eff380d3710 0x7eff380cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3cb9a700 1 -- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 msgr2=0x7eff380cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:04.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3cb9a700 1 --2- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff380cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3cb9a700 1 -- 172.21.15.84:0/1099118819 >> v1:172.21.15.107:6789/0 conn(0x7eff380cfa90 legacy=0x7eff3815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3cb9a700 1 -- 172.21.15.84:0/1099118819 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380b4e70 con 0x7eff380d3710 2023-07-06T21:40:04.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.787+0000 7eff3d39b700 1 --2- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff380cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:04.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.788+0000 7eff3cb9a700 1 --2- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380d3710 0x7eff380cf550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff28009840 tx=0x7eff28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed0b68d15bf34658 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:04.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.788+0000 7eff2ffff700 1 -- 172.21.15.84:0/1099118819 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28011830 con 0x7eff380d3710 2023-07-06T21:40:04.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.788+0000 7eff2ffff700 1 -- 172.21.15.84:0/1099118819 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff28011990 con 0x7eff380d3710 2023-07-06T21:40:04.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.788+0000 7eff2ffff700 1 -- 172.21.15.84:0/1099118819 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28011d20 con 0x7eff380d3710 2023-07-06T21:40:04.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.788+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380d3710 msgr2=0x7eff380cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.788+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380d3710 0x7eff380cf550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff28009840 tx=0x7eff28009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.789+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1099118819 shutdown_connections 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.789+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff380cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.789+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1099118819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380d3710 0x7eff380cf550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.789+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1099118819 >> 172.21.15.84:0/1099118819 conn(0x7eff3806b350 msgr2=0x7eff3806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.789+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1099118819 shutdown_connections 2023-07-06T21:40:04.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.789+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1099118819 wait complete. 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.790+0000 7eff3f5ff700 1 Processor -- start 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.790+0000 7eff3f5ff700 1 -- start start 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.791+0000 7eff3f5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff3815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.791+0000 7eff3f5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380cfa90 0x7eff38150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.791+0000 7eff3f5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 0x7eff38159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.791+0000 7eff3f5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3815e7e0 con 0x7eff380d3710 2023-07-06T21:40:04.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.791+0000 7eff3f5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff380c9f60 con 0x7eff380cfa90 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.791+0000 7eff3f5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff380ca240 con 0x7eff380b83c0 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 0x7eff38159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 0x7eff38159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57124/0 (socket says 172.21.15.84:57124) 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 -- 172.21.15.84:0/1252182257 learned_addr learned my addr 172.21.15.84:0/1252182257 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3d39b700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff3815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3cb9a700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380cfa90 0x7eff38150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 -- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 msgr2=0x7eff3815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff3815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 -- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380cfa90 msgr2=0x7eff38150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380cfa90 0x7eff38150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 -- 172.21.15.84:0/1252182257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff34009550 con 0x7eff380d3710 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.792+0000 7eff3db9c700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 0x7eff38159450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7eff3000cb50 tx=0x7eff3000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.793+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30003c00 con 0x7eff380d3710 2023-07-06T21:40:04.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.793+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff30003d60 con 0x7eff380d3710 2023-07-06T21:40:04.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.793+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3000aa20 con 0x7eff380d3710 2023-07-06T21:40:04.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.793+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff280094a0 con 0x7eff380d3710 2023-07-06T21:40:04.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.793+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff380ca800 con 0x7eff380d3710 2023-07-06T21:40:04.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.794+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff38160780 con 0x7eff380d3710 2023-07-06T21:40:04.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.795+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7eff3000ab80 con 0x7eff380d3710 2023-07-06T21:40:04.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.795+0000 7eff2dffb700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7eff1c06ace0 0x7eff1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:04.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.796+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7eff30089280 con 0x7eff380d3710 2023-07-06T21:40:04.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.796+0000 7eff3d39b700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7eff1c06ace0 0x7eff1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:04.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.796+0000 7eff3d39b700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7eff1c06ace0 0x7eff1c06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff34006d30 tx=0x7eff340092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:04.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.800+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7eff30054360 con 0x7eff380d3710 2023-07-06T21:40:04.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.997+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff380cfee0 con 0x7eff380d3710 2023-07-06T21:40:05.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:04.998+0000 7eff2dffb700 1 -- 172.21.15.84:0/1252182257 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18279 (secure 0 0 0) 0x7eff380cfee0 con 0x7eff380d3710 2023-07-06T21:40:05.001 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:05.002 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":23,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:39:27.429794+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:05.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7eff1c06ace0 msgr2=0x7eff1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7eff1c06ace0 0x7eff1c06d190 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7eff34006d30 tx=0x7eff340092c0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 msgr2=0x7eff38159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 0x7eff38159450 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7eff3000cb50 tx=0x7eff3000ce60 comp rx=0 tx=0).stop 2023-07-06T21:40:05.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 shutdown_connections 2023-07-06T21:40:05.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7eff1c06ace0 0x7eff1c06d190 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff380d3710 0x7eff38159450 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff380b83c0 0x7eff3815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 --2- 172.21.15.84:0/1252182257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff380cfa90 0x7eff38150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 >> 172.21.15.84:0/1252182257 conn(0x7eff3806b350 msgr2=0x7eff380b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:05.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 shutdown_connections 2023-07-06T21:40:05.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.001+0000 7eff3f5ff700 1 -- 172.21.15.84:0/1252182257 wait complete. 2023-07-06T21:40:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:05.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:05.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.215+0000 7f8c3fbff700 1 Processor -- start 2023-07-06T21:40:05.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.215+0000 7f8c3fbff700 1 -- start start 2023-07-06T21:40:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.216+0000 7f8c3fbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c380b81c0 0x7f8c380cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.216+0000 7f8c3fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b1a80 0x7f8c380cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.216+0000 7f8c3fbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c380ceda0 0x7f8c3815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.216+0000 7f8c3fbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c380d1830 con 0x7f8c380ceda0 2023-07-06T21:40:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3fbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c380cf330 con 0x7f8c380b81c0 2023-07-06T21:40:05.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380d1270 con 0x7f8c380b1a80 2023-07-06T21:40:05.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c380ceda0 0x7f8c3815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c380ceda0 0x7f8c3815ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57134/0 (socket says 172.21.15.84:57134) 2023-07-06T21:40:05.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 -- 172.21.15.84:0/2043609900 learned_addr learned my addr 172.21.15.84:0/2043609900 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:05.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3d99b700 1 --2- 172.21.15.84:0/2043609900 >> v2:172.21.15.107:3300/0 conn(0x7f8c380b81c0 0x7f8c380cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3d19a700 1 --2- 172.21.15.84:0/2043609900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b1a80 0x7f8c380cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 -- 172.21.15.84:0/2043609900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b1a80 msgr2=0x7f8c380cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:05.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 --2- 172.21.15.84:0/2043609900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b1a80 0x7f8c380cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 -- 172.21.15.84:0/2043609900 >> v2:172.21.15.107:3300/0 conn(0x7f8c380b81c0 msgr2=0x7f8c380cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 --2- 172.21.15.84:0/2043609900 >> v2:172.21.15.107:3300/0 conn(0x7f8c380b81c0 0x7f8c380cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.217+0000 7f8c3e19c700 1 -- 172.21.15.84:0/2043609900 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380d1550 con 0x7f8c380ceda0 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3e19c700 1 --2- 172.21.15.84:0/2043609900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c3815ff20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c380b2620 tx=0x7f8c3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af9d941e66652df server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3c999700 1 -- 172.21.15.84:0/2043609900 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000e070 con 0x7f8c380ceda0 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3c999700 1 -- 172.21.15.84:0/2043609900 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3000b910 con 0x7f8c380ceda0 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3c999700 1 -- 172.21.15.84:0/2043609900 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000bcc0 con 0x7f8c380ceda0 2023-07-06T21:40:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 -- 172.21.15.84:0/2043609900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 msgr2=0x7f8c3815ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/2043609900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c3815ff20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c380b2620 tx=0x7f8c3000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 -- 172.21.15.84:0/2043609900 shutdown_connections 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/2043609900 >> v2:172.21.15.107:3300/0 conn(0x7f8c380b81c0 0x7f8c380cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/2043609900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c3815ff20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/2043609900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b1a80 0x7f8c380cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.218+0000 7f8c3fbff700 1 -- 172.21.15.84:0/2043609900 >> 172.21.15.84:0/2043609900 conn(0x7f8c3806b2b0 msgr2=0x7f8c3806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:05.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.219+0000 7f8c3fbff700 1 -- 172.21.15.84:0/2043609900 shutdown_connections 2023-07-06T21:40:05.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.219+0000 7f8c3fbff700 1 -- 172.21.15.84:0/2043609900 wait complete. 2023-07-06T21:40:05.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.221+0000 7f8c3fbff700 1 Processor -- start 2023-07-06T21:40:05.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.221+0000 7f8c3fbff700 1 -- start start 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.222+0000 7f8c3fbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c380b1a80 0x7f8c380c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.222+0000 7f8c3fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b81c0 0x7f8c38157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.222+0000 7f8c3fbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c38157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.222+0000 7f8c3fbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380c8ad0 con 0x7f8c380ceda0 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.222+0000 7f8c3fbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380c8d70 con 0x7f8c380b1a80 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.222+0000 7f8c3fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c380c9050 con 0x7f8c380b81c0 2023-07-06T21:40:05.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c38157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c38157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57144/0 (socket says 172.21.15.84:57144) 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 -- 172.21.15.84:0/3524498227 learned_addr learned my addr 172.21.15.84:0/3524498227 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3d99b700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c380b1a80 0x7f8c380c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3d19a700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b81c0 0x7f8c38157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 -- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b81c0 msgr2=0x7f8c38157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b81c0 0x7f8c38157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 -- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c380b1a80 msgr2=0x7f8c380c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c380b1a80 0x7f8c380c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 -- 172.21.15.84:0/3524498227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009b30 con 0x7f8c380ceda0 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3d99b700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c380b1a80 0x7f8c380c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.223+0000 7f8c3e19c700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c38157360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3806bed0 tx=0x7f8c30004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.224+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300092e0 con 0x7f8c380ceda0 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.224+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c30004250 con 0x7f8c380ceda0 2023-07-06T21:40:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.224+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c30004600 con 0x7f8c380ceda0 2023-07-06T21:40:05.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.224+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c380c9330 con 0x7f8c380ceda0 2023-07-06T21:40:05.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.224+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c38158310 con 0x7f8c380ceda0 2023-07-06T21:40:05.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.225+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c380666c0 con 0x7f8c380ceda0 2023-07-06T21:40:05.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.226+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c30004760 con 0x7f8c380ceda0 2023-07-06T21:40:05.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.227+0000 7f8c2e7fc700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c1c06ace0 0x7f8c1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.227+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8c3008a620 con 0x7f8c380ceda0 2023-07-06T21:40:05.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.227+0000 7f8c3d99b700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c1c06ace0 0x7f8c1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.227+0000 7f8c3d99b700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c1c06ace0 0x7f8c1c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c380c9b00 tx=0x7f8c34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:05.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.231+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c300556c0 con 0x7f8c380ceda0 2023-07-06T21:40:05.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.425+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c380b5a20 con 0x7f8c380ceda0 2023-07-06T21:40:05.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.425+0000 7f8c2e7fc700 1 -- 172.21.15.84:0/3524498227 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c30058ce0 con 0x7f8c380ceda0 2023-07-06T21:40:05.428 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:05.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c1c06ace0 msgr2=0x7f8c1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c1c06ace0 0x7f8c1c06d190 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c380c9b00 tx=0x7f8c34008040 comp rx=0 tx=0).stop 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 msgr2=0x7f8c38157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c38157360 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c3806bed0 tx=0x7f8c30004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 shutdown_connections 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c1c06ace0 0x7f8c1c06d190 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c380ceda0 0x7f8c38157360 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c380b1a80 0x7f8c380c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 --2- 172.21.15.84:0/3524498227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c380b81c0 0x7f8c38157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 >> 172.21.15.84:0/3524498227 conn(0x7f8c3806b2b0 msgr2=0x7f8c380b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:05.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.427+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 shutdown_connections 2023-07-06T21:40:05.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.428+0000 7f8c3fbff700 1 -- 172.21.15.84:0/3524498227 wait complete. 2023-07-06T21:40:05.448 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:05.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.642+0000 7f2a931ff700 1 Processor -- start 2023-07-06T21:40:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.643+0000 7f2a931ff700 1 -- start start 2023-07-06T21:40:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a931ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b81c0 0x7f2a8c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a931ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c0d1830 con 0x7f2a8c0ceda0 2023-07-06T21:40:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a931ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c160d40 con 0x7f2a8c0b1a80 2023-07-06T21:40:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a931ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cf330 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a9179c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2a8c0ceda0 0x7f2a8c160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55934/0 (socket says 172.21.15.84:55934) 2023-07-06T21:40:05.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a9179c700 1 -- 172.21.15.84:0/2228257200 learned_addr learned my addr 172.21.15.84:0/2228257200 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:05.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a90f9b700 1 --2- 172.21.15.84:0/2228257200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b81c0 0x7f2a8c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888085518 0 0) 0x7f2a8c0d1830 con 0x7f2a8c0ceda0 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.644+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a7c0035b0 con 0x7f2a8c0ceda0 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817893657 0 0) 0x7f2a8c160d40 con 0x7f2a8c0b1a80 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a8c0d1830 con 0x7f2a8c0b1a80 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a90f9b700 1 -- 172.21.15.84:0/2228257200 >> v1:172.21.15.107:6789/0 conn(0x7f2a8c0b1a80 legacy=0x7f2a8c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a90f9b700 1 -- 172.21.15.84:0/2228257200 >> v1:172.21.15.84:6789/0 conn(0x7f2a8c0ceda0 legacy=0x7f2a8c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a90f9b700 1 -- 172.21.15.84:0/2228257200 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0d1270 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a90f9b700 1 --2- 172.21.15.84:0/2228257200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b81c0 0x7f2a8c0cf600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009870 tx=0x7f2a80009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318bf88df2a7392b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80011830 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a80011990 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.645+0000 7f2a8b7fe700 1 -- 172.21.15.84:0/2228257200 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80011d20 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 -- 172.21.15.84:0/2228257200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b81c0 msgr2=0x7f2a8c0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2228257200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b81c0 0x7f2a8c0cf600 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009870 tx=0x7f2a80009b80 comp rx=0 tx=0).stop 2023-07-06T21:40:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 -- 172.21.15.84:0/2228257200 shutdown_connections 2023-07-06T21:40:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2228257200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b81c0 0x7f2a8c0cf600 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 -- 172.21.15.84:0/2228257200 >> 172.21.15.84:0/2228257200 conn(0x7f2a8c06b2b0 msgr2=0x7f2a8c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 -- 172.21.15.84:0/2228257200 shutdown_connections 2023-07-06T21:40:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.646+0000 7f2a931ff700 1 -- 172.21.15.84:0/2228257200 wait complete. 2023-07-06T21:40:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.648+0000 7f2a931ff700 1 Processor -- start 2023-07-06T21:40:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.648+0000 7f2a931ff700 1 -- start start 2023-07-06T21:40:05.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a931ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b1a80 0x7f2a8c155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a931ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 0x7f2a8c0c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a931ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0ceda0 0x7f2a8c146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a931ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cc0b0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a931ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cb7c0 con 0x7f2a8c0ceda0 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a931ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cba60 con 0x7f2a8c0b1a80 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 0x7f2a8c0c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 0x7f2a8c0c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57154/0 (socket says 172.21.15.84:57154) 2023-07-06T21:40:05.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 -- 172.21.15.84:0/2732858984 learned_addr learned my addr 172.21.15.84:0/2732858984 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a90f9b700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b1a80 0x7f2a8c155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a9179c700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0ceda0 0x7f2a8c146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 -- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b1a80 msgr2=0x7f2a8c155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b1a80 0x7f2a8c155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 -- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0ceda0 msgr2=0x7f2a8c146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0ceda0 0x7f2a8c146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.649+0000 7f2a8bfff700 1 -- 172.21.15.84:0/2732858984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800094a0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.650+0000 7f2a8bfff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 0x7f2a8c0c91f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c520 tx=0x7f2a7800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.650+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780052d0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.650+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78005430 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.650+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c0cbd40 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.650+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780038c0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.650+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c158cf0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.651+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c0cc220 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.652+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a78003a20 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.653+0000 7f2a897fa700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a7006ace0 0x7f2a7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.653+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2a780889f0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.653+0000 7f2a90f9b700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a7006ace0 0x7f2a7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:05.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.654+0000 7f2a90f9b700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a7006ace0 0x7f2a7006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2a800097c0 tx=0x7f2a8001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:05.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.657+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a78053970 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.851+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2a8c04e1d0 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.852+0000 7f2a897fa700 1 -- 172.21.15.84:0/2732858984 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2a78017020 con 0x7f2a8c0b81c0 2023-07-06T21:40:05.855 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:05.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a7006ace0 msgr2=0x7f2a7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a7006ace0 0x7f2a7006d190 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2a800097c0 tx=0x7f2a8001c040 comp rx=0 tx=0).stop 2023-07-06T21:40:05.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 msgr2=0x7f2a8c0c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 0x7f2a8c0c91f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c520 tx=0x7f2a7800c830 comp rx=0 tx=0).stop 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 shutdown_connections 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a7006ace0 0x7f2a7006d190 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0ceda0 0x7f2a8c146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c0b1a80 0x7f2a8c155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 --2- 172.21.15.84:0/2732858984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a8c0b81c0 0x7f2a8c0c91f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:05.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 >> 172.21.15.84:0/2732858984 conn(0x7f2a8c06b2b0 msgr2=0x7f2a8c0b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:05.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 shutdown_connections 2023-07-06T21:40:05.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:05.854+0000 7f2a931ff700 1 -- 172.21.15.84:0/2732858984 wait complete. 2023-07-06T21:40:05.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.071+0000 7f42b61ff700 1 Processor -- start 2023-07-06T21:40:06.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.071+0000 7f42b61ff700 1 -- start start 2023-07-06T21:40:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.072+0000 7f42b61ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42b00b81c0 0x7f42b00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.072+0000 7f42b61ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42b61ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b00d1830 con 0x7f42b00b1a80 2023-07-06T21:40:06.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42b61ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b00cf330 con 0x7f42b00b81c0 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42b61ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00d1270 con 0x7f42b00ceda0 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42b00b81c0 0x7f42b00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f42b00b81c0 0x7f42b00cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57158/0 (socket says 172.21.15.84:57158) 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 -- 172.21.15.84:0/2930738292 learned_addr learned my addr 172.21.15.84:0/2930738292 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42affff700 1 --2- 172.21.15.84:0/2930738292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 -- 172.21.15.84:0/2930738292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 msgr2=0x7f42b00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:06.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 --2- 172.21.15.84:0/2930738292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 -- 172.21.15.84:0/2930738292 >> v1:172.21.15.84:6789/0 conn(0x7f42b00b1a80 legacy=0x7f42b015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 -- 172.21.15.84:0/2930738292 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b00d1550 con 0x7f42b00b81c0 2023-07-06T21:40:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.073+0000 7f42af7fe700 1 --2- 172.21.15.84:0/2930738292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00cf600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42a00098f0 tx=0x7f42a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d368945fa1f674 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42ae7fc700 1 -- 172.21.15.84:0/2930738292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000b8a0 con 0x7f42b00b81c0 2023-07-06T21:40:06.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42ae7fc700 1 -- 172.21.15.84:0/2930738292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a000ba00 con 0x7f42b00b81c0 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42ae7fc700 1 -- 172.21.15.84:0/2930738292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000bd90 con 0x7f42b00b81c0 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 -- 172.21.15.84:0/2930738292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 msgr2=0x7f42b00cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2930738292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00cf600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42a00098f0 tx=0x7f42a0009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 -- 172.21.15.84:0/2930738292 shutdown_connections 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2930738292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2930738292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00cf600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 -- 172.21.15.84:0/2930738292 >> 172.21.15.84:0/2930738292 conn(0x7f42b006b2b0 msgr2=0x7f42b006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:06.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 -- 172.21.15.84:0/2930738292 shutdown_connections 2023-07-06T21:40:06.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.074+0000 7f42b61ff700 1 -- 172.21.15.84:0/2930738292 wait complete. 2023-07-06T21:40:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.076+0000 7f42b61ff700 1 Processor -- start 2023-07-06T21:40:06.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.076+0000 7f42b61ff700 1 -- start start 2023-07-06T21:40:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42b61ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42b00b1a80 0x7f42b014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42b61ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42b61ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42b61ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b01e8250 con 0x7f42b00b81c0 2023-07-06T21:40:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42b61ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b014e580 con 0x7f42b00b1a80 2023-07-06T21:40:06.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42b61ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b00c68c0 con 0x7f42b00ceda0 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42aeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42aeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00c8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57174/0 (socket says 172.21.15.84:57174) 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.077+0000 7f42aeffd700 1 -- 172.21.15.84:0/2328565541 learned_addr learned my addr 172.21.15.84:0/2328565541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42affff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42aeffd700 1 -- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 msgr2=0x7f42b00c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42af7fe700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42b00b1a80 0x7f42b014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42aeffd700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42aeffd700 1 -- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42b00b1a80 msgr2=0x7f42b014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42aeffd700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42b00b1a80 0x7f42b014e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42aeffd700 1 -- 172.21.15.84:0/2328565541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a0009550 con 0x7f42b00b81c0 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42affff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42af7fe700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42b00b1a80 0x7f42b014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42aeffd700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00c8f60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c450 tx=0x7f42a400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40052d0 con 0x7f42b00b81c0 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4005430 con 0x7f42b00b81c0 2023-07-06T21:40:06.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40038c0 con 0x7f42b00b81c0 2023-07-06T21:40:06.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b00c6ba0 con 0x7f42b00b81c0 2023-07-06T21:40:06.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.078+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b00c7160 con 0x7f42b00b81c0 2023-07-06T21:40:06.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.079+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b00c8800 con 0x7f42b00b81c0 2023-07-06T21:40:06.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.081+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f42a4003a20 con 0x7f42b00b81c0 2023-07-06T21:40:06.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.081+0000 7f4297fff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f429c06acb0 0x7f429c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.082+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f42a4088820 con 0x7f42b00b81c0 2023-07-06T21:40:06.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.082+0000 7f42af7fe700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f429c06acb0 0x7f429c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.082+0000 7f42af7fe700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f429c06acb0 0x7f429c06d160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f42a00098f0 tx=0x7f42a0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.086+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f42b00c8800 con 0x7f42b00b81c0 2023-07-06T21:40:06.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.286+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42b00c8800 con 0x7f42b00b81c0 2023-07-06T21:40:06.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.287+0000 7f4297fff700 1 -- 172.21.15.84:0/2328565541 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f42b00c8800 con 0x7f42b00b81c0 2023-07-06T21:40:06.290 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:06.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f429c06acb0 msgr2=0x7f429c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f429c06acb0 0x7f429c06d160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f42a00098f0 tx=0x7f42a0008030 comp rx=0 tx=0).stop 2023-07-06T21:40:06.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 msgr2=0x7f42b00c8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00c8f60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c450 tx=0x7f42a400c810 comp rx=0 tx=0).stop 2023-07-06T21:40:06.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 shutdown_connections 2023-07-06T21:40:06.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f429c06acb0 0x7f429c06d160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42b00ceda0 0x7f42b00c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42b00b1a80 0x7f42b014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 --2- 172.21.15.84:0/2328565541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42b00b81c0 0x7f42b00c8f60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 >> 172.21.15.84:0/2328565541 conn(0x7f42b006b2b0 msgr2=0x7f42b00d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 shutdown_connections 2023-07-06T21:40:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.289+0000 7f42b61ff700 1 -- 172.21.15.84:0/2328565541 wait complete. 2023-07-06T21:40:06.309 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:06.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.508+0000 7f4b56fff700 1 Processor -- start 2023-07-06T21:40:06.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.508+0000 7f4b56fff700 1 -- start start 2023-07-06T21:40:06.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b56fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4b500b81c0 0x7f4b500cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b56fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b500b1a80 0x7f4b500cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b56fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b500d1830 con 0x7f4b500ceda0 2023-07-06T21:40:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b56fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500cf330 con 0x7f4b500b1a80 2023-07-06T21:40:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b56fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b500d1270 con 0x7f4b500b81c0 2023-07-06T21:40:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b4ffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4b500b1a80 0x7f4b500cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b5559c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4b500ceda0 0x7f4b5015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55942/0 (socket says 172.21.15.84:55942) 2023-07-06T21:40:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b5559c700 1 -- 172.21.15.84:0/791320975 learned_addr learned my addr 172.21.15.84:0/791320975 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:06.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.509+0000 7f4b54d9b700 1 --2- 172.21.15.84:0/791320975 >> v2:172.21.15.107:3300/0 conn(0x7f4b500b81c0 0x7f4b500cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4ffff700 1 -- 172.21.15.84:0/791320975 >> v2:172.21.15.107:3300/0 conn(0x7f4b500b81c0 msgr2=0x7f4b500cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:06.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4ffff700 1 --2- 172.21.15.84:0/791320975 >> v2:172.21.15.107:3300/0 conn(0x7f4b500b81c0 0x7f4b500cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4ffff700 1 -- 172.21.15.84:0/791320975 >> v1:172.21.15.84:6789/0 conn(0x7f4b500ceda0 legacy=0x7f4b5015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4ffff700 1 -- 172.21.15.84:0/791320975 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b500d1550 con 0x7f4b500b1a80 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4f7fe700 1 -- 172.21.15.84:0/791320975 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2297473812 0 0) 0x7f4b500d1830 con 0x7f4b500ceda0 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4ffff700 1 --2- 172.21.15.84:0/791320975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500b1a80 0x7f4b500cfb40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b500b2620 tx=0x7f4b3c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc8c902cee7fc4e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4f7fe700 1 -- 172.21.15.84:0/791320975 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c011830 con 0x7f4b500b1a80 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4f7fe700 1 -- 172.21.15.84:0/791320975 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b3c011990 con 0x7f4b500b1a80 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.510+0000 7f4b4f7fe700 1 -- 172.21.15.84:0/791320975 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c011d20 con 0x7f4b500b1a80 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 -- 172.21.15.84:0/791320975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500b1a80 msgr2=0x7f4b500cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 --2- 172.21.15.84:0/791320975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500b1a80 0x7f4b500cfb40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4b500b2620 tx=0x7f4b3c009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:06.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 -- 172.21.15.84:0/791320975 shutdown_connections 2023-07-06T21:40:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 --2- 172.21.15.84:0/791320975 >> v2:172.21.15.107:3300/0 conn(0x7f4b500b81c0 0x7f4b500cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 --2- 172.21.15.84:0/791320975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500b1a80 0x7f4b500cfb40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 -- 172.21.15.84:0/791320975 >> 172.21.15.84:0/791320975 conn(0x7f4b5006b2b0 msgr2=0x7f4b5006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 -- 172.21.15.84:0/791320975 shutdown_connections 2023-07-06T21:40:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.511+0000 7f4b56fff700 1 -- 172.21.15.84:0/791320975 wait complete. 2023-07-06T21:40:06.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.512+0000 7f4b56fff700 1 Processor -- start 2023-07-06T21:40:06.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 -- start start 2023-07-06T21:40:06.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b500b1a80 0x7f4b501f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b500b81c0 0x7f4b5014a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 0x7f4b501485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.519 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c8490 con 0x7f4b500ceda0 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b500c9870 con 0x7f4b500b1a80 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.513+0000 7f4b56fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b50147520 con 0x7f4b500b81c0 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 0x7f4b501485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 0x7f4b501485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57192/0 (socket says 172.21.15.84:57192) 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 -- 172.21.15.84:0/3642321949 learned_addr learned my addr 172.21.15.84:0/3642321949 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b54d9b700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b500b1a80 0x7f4b501f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.520 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b4ffff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b500b81c0 0x7f4b5014a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 -- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b500b81c0 msgr2=0x7f4b5014a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b500b81c0 0x7f4b5014a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 -- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b500b1a80 msgr2=0x7f4b501f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b500b1a80 0x7f4b501f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 -- 172.21.15.84:0/3642321949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c0094a0 con 0x7f4b500ceda0 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.514+0000 7f4b5559c700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 0x7f4b501485c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cca0 tx=0x7f4b4800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.515+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800cea0 con 0x7f4b500ceda0 2023-07-06T21:40:06.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.515+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4800ade0 con 0x7f4b500ceda0 2023-07-06T21:40:06.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.515+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b50147800 con 0x7f4b500ceda0 2023-07-06T21:40:06.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.515+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48007310 con 0x7f4b500ceda0 2023-07-06T21:40:06.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.515+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b50147dc0 con 0x7f4b500ceda0 2023-07-06T21:40:06.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.516+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b500b53e0 con 0x7f4b500ceda0 2023-07-06T21:40:06.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.517+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4b48007470 con 0x7f4b500ceda0 2023-07-06T21:40:06.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.518+0000 7f4b4d7fa700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4b3406ace0 0x7f4b3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.518+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4b48089210 con 0x7f4b500ceda0 2023-07-06T21:40:06.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.518+0000 7f4b54d9b700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4b3406ace0 0x7f4b3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.518+0000 7f4b54d9b700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4b3406ace0 0x7f4b3406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009b70 tx=0x7f4b44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.522+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4b48054220 con 0x7f4b500ceda0 2023-07-06T21:40:06.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.720+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4b5004e1d0 con 0x7f4b500ceda0 2023-07-06T21:40:06.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.721+0000 7f4b4d7fa700 1 -- 172.21.15.84:0/3642321949 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4b48057840 con 0x7f4b500ceda0 2023-07-06T21:40:06.723 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:06.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4b3406ace0 msgr2=0x7f4b3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4b3406ace0 0x7f4b3406d190 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009b70 tx=0x7f4b44008040 comp rx=0 tx=0).stop 2023-07-06T21:40:06.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 msgr2=0x7f4b501485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 0x7f4b501485c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800cca0 tx=0x7f4b4800a340 comp rx=0 tx=0).stop 2023-07-06T21:40:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 shutdown_connections 2023-07-06T21:40:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4b3406ace0 0x7f4b3406d190 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4b500ceda0 0x7f4b501485c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b500b1a80 0x7f4b501f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 --2- 172.21.15.84:0/3642321949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b500b81c0 0x7f4b5014a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 >> 172.21.15.84:0/3642321949 conn(0x7f4b5006b2b0 msgr2=0x7f4b500b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:06.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 shutdown_connections 2023-07-06T21:40:06.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.723+0000 7f4b56fff700 1 -- 172.21.15.84:0/3642321949 wait complete. 2023-07-06T21:40:06.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:06.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.938+0000 7f1b201ff700 1 Processor -- start 2023-07-06T21:40:06.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.938+0000 7f1b201ff700 1 -- start start 2023-07-06T21:40:06.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b201ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b201ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180cd910 con 0x7f1b180cfa90 2023-07-06T21:40:06.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b201ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180cfee0 con 0x7f1b180b83c0 2023-07-06T21:40:06.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b201ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180d0a80 con 0x7f1b180d3710 2023-07-06T21:40:06.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b1e79c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1b180cfa90 0x7f1b180d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55950/0 (socket says 172.21.15.84:55950) 2023-07-06T21:40:06.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b1e79c700 1 -- 172.21.15.84:0/3924452763 learned_addr learned my addr 172.21.15.84:0/3924452763 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:06.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.939+0000 7f1b1d79a700 1 --2- 172.21.15.84:0/3924452763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288535474 0 0) 0x7f1b10001490 con 0x7f1b180cfa90 2023-07-06T21:40:06.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b040035b0 con 0x7f1b180cfa90 2023-07-06T21:40:06.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877894701 0 0) 0x7f1b14001110 con 0x7f1b180b83c0 2023-07-06T21:40:06.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b10001490 con 0x7f1b180b83c0 2023-07-06T21:40:06.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3924452763 >> v1:172.21.15.107:6789/0 conn(0x7f1b180b83c0 legacy=0x7f1b180cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3924452763 >> v1:172.21.15.84:6789/0 conn(0x7f1b180cfa90 legacy=0x7f1b180d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3924452763 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180b8790 con 0x7f1b180d3710 2023-07-06T21:40:06.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1d79a700 1 --2- 172.21.15.84:0/3924452763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180cf550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009870 tx=0x7f1b08009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dba2f5abe0f5c94 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011830 con 0x7f1b180d3710 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08011990 con 0x7f1b180d3710 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.940+0000 7f1b1cf99700 1 -- 172.21.15.84:0/3924452763 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011d20 con 0x7f1b180d3710 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 -- 172.21.15.84:0/3924452763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 msgr2=0x7f1b180cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3924452763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180cf550 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b08009870 tx=0x7f1b08009b80 comp rx=0 tx=0).stop 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 -- 172.21.15.84:0/3924452763 shutdown_connections 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3924452763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180cf550 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 -- 172.21.15.84:0/3924452763 >> 172.21.15.84:0/3924452763 conn(0x7f1b1806b350 msgr2=0x7f1b1806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:06.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 -- 172.21.15.84:0/3924452763 shutdown_connections 2023-07-06T21:40:06.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.941+0000 7f1b201ff700 1 -- 172.21.15.84:0/3924452763 wait complete. 2023-07-06T21:40:06.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.943+0000 7f1b201ff700 1 Processor -- start 2023-07-06T21:40:06.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.943+0000 7f1b201ff700 1 -- start start 2023-07-06T21:40:06.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b201ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b180b83c0 0x7f1b18153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b201ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 0x7f1b181e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b201ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b201ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b18153f50 con 0x7f1b180cfa90 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b201ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b181e7670 con 0x7f1b180b83c0 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b201ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b181e7950 con 0x7f1b180d3710 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 0x7f1b181e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 0x7f1b181e6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57196/0 (socket says 172.21.15.84:57196) 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3013740081 learned_addr learned my addr 172.21.15.84:0/3013740081 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:06.953 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3013740081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 msgr2=0x7f1b180b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3013740081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b180b83c0 msgr2=0x7f1b18153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b180b83c0 0x7f1b18153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.944+0000 7f1b1d79a700 1 -- 172.21.15.84:0/3013740081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080094a0 con 0x7f1b180cfa90 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.945+0000 7f1b1d79a700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 0x7f1b181e6e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b080090b0 tx=0x7f1b08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.945+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0801a860 con 0x7f1b180cfa90 2023-07-06T21:40:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.945+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0801a9c0 con 0x7f1b180cfa90 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.945+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b181e7c30 con 0x7f1b180cfa90 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.945+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0801ad30 con 0x7f1b180cfa90 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.945+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b181e81f0 con 0x7f1b180cfa90 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.946+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1815baa0 con 0x7f1b180cfa90 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.947+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1b08022070 con 0x7f1b180cfa90 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.948+0000 7f1b0e7fc700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1afc06ace0 0x7f1afc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.948+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1b0808a630 con 0x7f1b180cfa90 2023-07-06T21:40:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.948+0000 7f1b1df9b700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1afc06ace0 0x7f1afc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.949+0000 7f1b1df9b700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1afc06ace0 0x7f1afc06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b14000f60 tx=0x7f1b14001270 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:06.952+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1b080556d0 con 0x7f1b180cfa90 2023-07-06T21:40:07.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.152+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b1815b7c0 con 0x7f1b180cfa90 2023-07-06T21:40:07.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.153+0000 7f1b0e7fc700 1 -- 172.21.15.84:0/3013740081 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f1b08006cc0 con 0x7f1b180cfa90 2023-07-06T21:40:07.156 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:07.156 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:06.966630+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:07.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1afc06ace0 msgr2=0x7f1afc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1afc06ace0 0x7f1afc06d190 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1b14000f60 tx=0x7f1b14001270 comp rx=0 tx=0).stop 2023-07-06T21:40:07.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 msgr2=0x7f1b181e6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 0x7f1b181e6e20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1b080090b0 tx=0x7f1b08004750 comp rx=0 tx=0).stop 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 shutdown_connections 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1afc06ace0 0x7f1afc06d190 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1b180d3710 0x7f1b180b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1b180b83c0 0x7f1b18153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 --2- 172.21.15.84:0/3013740081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b180cfa90 0x7f1b181e6e20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 >> 172.21.15.84:0/3013740081 conn(0x7f1b1806b350 msgr2=0x7f1b1815a950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 shutdown_connections 2023-07-06T21:40:07.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.156+0000 7f1b201ff700 1 -- 172.21.15.84:0/3013740081 wait complete. 2023-07-06T21:40:07.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:07.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:07.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.373+0000 7f3f9e1ff700 1 Processor -- start 2023-07-06T21:40:07.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.373+0000 7f3f9e1ff700 1 -- start start 2023-07-06T21:40:07.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.374+0000 7f3f9e1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f980b81c0 0x7f3f980cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f9e1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f980b1a80 0x7f3f980cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f9e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f9815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f9e1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980d1830 con 0x7f3f980b81c0 2023-07-06T21:40:07.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f9e1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980cf330 con 0x7f3f980b1a80 2023-07-06T21:40:07.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f9e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980d1270 con 0x7f3f980ceda0 2023-07-06T21:40:07.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f980b81c0 0x7f3f980cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f980b81c0 0x7f3f980cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57212/0 (socket says 172.21.15.84:57212) 2023-07-06T21:40:07.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 -- 172.21.15.84:0/3671913231 learned_addr learned my addr 172.21.15.84:0/3671913231 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:07.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 -- 172.21.15.84:0/3671913231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 msgr2=0x7f3f9815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f97fff700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f9815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f96ffd700 1 --2- 172.21.15.84:0/3671913231 >> v2:172.21.15.107:3300/0 conn(0x7f3f980b1a80 0x7f3f980cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f9815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 -- 172.21.15.84:0/3671913231 >> v2:172.21.15.107:3300/0 conn(0x7f3f980b1a80 msgr2=0x7f3f980cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 --2- 172.21.15.84:0/3671913231 >> v2:172.21.15.107:3300/0 conn(0x7f3f980b1a80 0x7f3f980cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f977fe700 1 -- 172.21.15.84:0/3671913231 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980d1550 con 0x7f3f980b81c0 2023-07-06T21:40:07.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f96ffd700 1 --2- 172.21.15.84:0/3671913231 >> v2:172.21.15.107:3300/0 conn(0x7f3f980b1a80 0x7f3f980cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:07.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.375+0000 7f3f97fff700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f9815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:07.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.376+0000 7f3f977fe700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980cf600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f980d0770 tx=0x7f3f88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=932366cd25d900ba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:07.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.376+0000 7f3f967fc700 1 -- 172.21.15.84:0/3671913231 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800b870 con 0x7f3f980b81c0 2023-07-06T21:40:07.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.376+0000 7f3f967fc700 1 -- 172.21.15.84:0/3671913231 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8800b9d0 con 0x7f3f980b81c0 2023-07-06T21:40:07.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.376+0000 7f3f967fc700 1 -- 172.21.15.84:0/3671913231 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8800bd60 con 0x7f3f980b81c0 2023-07-06T21:40:07.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.376+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/3671913231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 msgr2=0x7f3f980cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.376+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980cf600 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3f980d0770 tx=0x7f3f88009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/3671913231 shutdown_connections 2023-07-06T21:40:07.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f9815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/3671913231 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980cf600 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/3671913231 >> v2:172.21.15.107:3300/0 conn(0x7f3f980b1a80 0x7f3f980cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/3671913231 >> 172.21.15.84:0/3671913231 conn(0x7f3f9806b2b0 msgr2=0x7f3f9806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:07.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/3671913231 shutdown_connections 2023-07-06T21:40:07.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.377+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/3671913231 wait complete. 2023-07-06T21:40:07.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.379+0000 7f3f9e1ff700 1 Processor -- start 2023-07-06T21:40:07.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.379+0000 7f3f9e1ff700 1 -- start start 2023-07-06T21:40:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.380+0000 7f3f9e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f980b1a80 0x7f3f980b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.380+0000 7f3f9e1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.380+0000 7f3f9e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f98155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.380+0000 7f3f9e1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980b3ef0 con 0x7f3f980b81c0 2023-07-06T21:40:07.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.380+0000 7f3f9e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980ca1f0 con 0x7f3f980b1a80 2023-07-06T21:40:07.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.380+0000 7f3f9e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f981f3000 con 0x7f3f980ceda0 2023-07-06T21:40:07.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57224/0 (socket says 172.21.15.84:57224) 2023-07-06T21:40:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 -- 172.21.15.84:0/2012445359 learned_addr learned my addr 172.21.15.84:0/2012445359 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f977fe700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f980b1a80 0x7f3f980b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f97fff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f98155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 -- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 msgr2=0x7f3f98155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f98155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 -- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f980b1a80 msgr2=0x7f3f980b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f980b1a80 0x7f3f980b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 -- 172.21.15.84:0/2012445359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f88009550 con 0x7f3f980b81c0 2023-07-06T21:40:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.381+0000 7f3f96ffd700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980ca8c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00c750 tx=0x7f3f8c00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.382+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c011800 con 0x7f3f980b81c0 2023-07-06T21:40:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.382+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c011960 con 0x7f3f980b81c0 2023-07-06T21:40:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.382+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f981f32e0 con 0x7f3f980b81c0 2023-07-06T21:40:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.382+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c011cd0 con 0x7f3f980b81c0 2023-07-06T21:40:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.382+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f980cae00 con 0x7f3f980b81c0 2023-07-06T21:40:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.383+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f980caff0 con 0x7f3f980b81c0 2023-07-06T21:40:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.384+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f8c021350 con 0x7f3f980b81c0 2023-07-06T21:40:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.385+0000 7f3f7ffff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f8406ace0 0x7f3f8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.385+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3f8c0893f0 con 0x7f3f980b81c0 2023-07-06T21:40:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.385+0000 7f3f977fe700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f8406ace0 0x7f3f8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.385+0000 7f3f977fe700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f8406ace0 0x7f3f8406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f88000c00 tx=0x7f3f88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.389+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f8c057ee0 con 0x7f3f980b81c0 2023-07-06T21:40:07.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.586+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f980cdeb0 con 0x7f3f980b81c0 2023-07-06T21:40:07.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.586+0000 7f3f7ffff700 1 -- 172.21.15.84:0/2012445359 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f8c057a70 con 0x7f3f980b81c0 2023-07-06T21:40:07.589 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:07.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.588+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f8406ace0 msgr2=0x7f3f8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.588+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f8406ace0 0x7f3f8406d190 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3f88000c00 tx=0x7f3f88008030 comp rx=0 tx=0).stop 2023-07-06T21:40:07.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.588+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 msgr2=0x7f3f980ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.588+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980ca8c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c00c750 tx=0x7f3f8c00ca60 comp rx=0 tx=0).stop 2023-07-06T21:40:07.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 shutdown_connections 2023-07-06T21:40:07.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f8406ace0 0x7f3f8406d190 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f980ceda0 0x7f3f98155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f980b1a80 0x7f3f980b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 --2- 172.21.15.84:0/2012445359 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f980b81c0 0x7f3f980ca8c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 >> 172.21.15.84:0/2012445359 conn(0x7f3f9806b2b0 msgr2=0x7f3f980d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:07.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 shutdown_connections 2023-07-06T21:40:07.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.589+0000 7f3f9e1ff700 1 -- 172.21.15.84:0/2012445359 wait complete. 2023-07-06T21:40:07.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:07.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.802+0000 7fdf109ff700 1 Processor -- start 2023-07-06T21:40:07.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.802+0000 7fdf109ff700 1 -- start start 2023-07-06T21:40:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf109ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf0c0cfd10 0x7fdf0c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf109ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0d1830 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf109ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c0cf330 con 0x7fdf0c0b1a80 2023-07-06T21:40:07.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf109ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c0d1270 con 0x7fdf0c0cfd10 2023-07-06T21:40:07.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf09d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdf0c0cfd10 0x7fdf0c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf0a59c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdf0c0b81c0 0x7fdf0c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55962/0 (socket says 172.21.15.84:55962) 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.803+0000 7fdf0a59c700 1 -- 172.21.15.84:0/1697473983 learned_addr learned my addr 172.21.15.84:0/1697473983 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf09d9b700 1 -- 172.21.15.84:0/1697473983 >> v1:172.21.15.107:6789/0 conn(0x7fdf0c0b1a80 legacy=0x7fdf0c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf09d9b700 1 -- 172.21.15.84:0/1697473983 >> v1:172.21.15.84:6789/0 conn(0x7fdf0c0b81c0 legacy=0x7fdf0c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf0959a700 1 -- 172.21.15.84:0/1697473983 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609851960 0 0) 0x7fdf0c0d1830 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf09d9b700 1 -- 172.21.15.84:0/1697473983 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0d1550 con 0x7fdf0c0cfd10 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf09d9b700 1 --2- 172.21.15.84:0/1697473983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0cfd10 0x7fdf0c0d2380 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009840 tx=0x7fdf00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7b21569da83b8d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:07.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf0959a700 1 -- 172.21.15.84:0/1697473983 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00011830 con 0x7fdf0c0cfd10 2023-07-06T21:40:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf0959a700 1 -- 172.21.15.84:0/1697473983 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf00011990 con 0x7fdf0c0cfd10 2023-07-06T21:40:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.804+0000 7fdf0959a700 1 -- 172.21.15.84:0/1697473983 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00011d20 con 0x7fdf0c0cfd10 2023-07-06T21:40:07.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 -- 172.21.15.84:0/1697473983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0cfd10 msgr2=0x7fdf0c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 --2- 172.21.15.84:0/1697473983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0cfd10 0x7fdf0c0d2380 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009840 tx=0x7fdf00009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 -- 172.21.15.84:0/1697473983 shutdown_connections 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 --2- 172.21.15.84:0/1697473983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0cfd10 0x7fdf0c0d2380 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 -- 172.21.15.84:0/1697473983 >> 172.21.15.84:0/1697473983 conn(0x7fdf0c06b2b0 msgr2=0x7fdf0c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 -- 172.21.15.84:0/1697473983 shutdown_connections 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.805+0000 7fdf109ff700 1 -- 172.21.15.84:0/1697473983 wait complete. 2023-07-06T21:40:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.807+0000 7fdf109ff700 1 Processor -- start 2023-07-06T21:40:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.807+0000 7fdf109ff700 1 -- start start 2023-07-06T21:40:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf109ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0b1a80 0x7fdf0c0c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf109ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 0x7fdf0c15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf109ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c0cfd10 0x7fdf0c1e7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf109ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c15baa0 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf109ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0c77c0 con 0x7fdf0c0b1a80 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf109ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c0c7aa0 con 0x7fdf0c0cfd10 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf09d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 0x7fdf0c15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf09d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 0x7fdf0c15b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57242/0 (socket says 172.21.15.84:57242) 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf09d9b700 1 -- 172.21.15.84:0/711226648 learned_addr learned my addr 172.21.15.84:0/711226648 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf0a59c700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0b1a80 0x7fdf0c0c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf0ad9d700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c0cfd10 0x7fdf0c1e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.808+0000 7fdf09d9b700 1 -- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c0cfd10 msgr2=0x7fdf0c1e7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf09d9b700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c0cfd10 0x7fdf0c1e7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf09d9b700 1 -- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0b1a80 msgr2=0x7fdf0c0c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf09d9b700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0b1a80 0x7fdf0c0c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf09d9b700 1 -- 172.21.15.84:0/711226648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf000094a0 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf0a59c700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0b1a80 0x7fdf0c0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:07.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf09d9b700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 0x7fdf0c15b560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf00014040 tx=0x7fdf00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000036a0 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0001f070 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c0c7d80 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00003a40 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.809+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1e1d90 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.810+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c1e1bf0 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.811+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdf00003ba0 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.813+0000 7fdefaffd700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdeec06acb0 0x7fdeec06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:07.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.813+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdf00021070 con 0x7fdf0c0b81c0 2023-07-06T21:40:07.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.813+0000 7fdf0a59c700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdeec06acb0 0x7fdeec06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:07.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.813+0000 7fdf0a59c700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdeec06acb0 0x7fdeec06d160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000f60 tx=0x7fdefc007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:07.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:07.817+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdf0c1e1bf0 con 0x7fdf0c0b81c0 2023-07-06T21:40:08.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.015+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf0c1e1bf0 con 0x7fdf0c0b81c0 2023-07-06T21:40:08.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.015+0000 7fdefaffd700 1 -- 172.21.15.84:0/711226648 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf0c1e1bf0 con 0x7fdf0c0b81c0 2023-07-06T21:40:08.018 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:08.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdeec06acb0 msgr2=0x7fdeec06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdeec06acb0 0x7fdeec06d160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdefc000f60 tx=0x7fdefc007ce0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 msgr2=0x7fdf0c15b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 0x7fdf0c15b560 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf00014040 tx=0x7fdf00004750 comp rx=0 tx=0).stop 2023-07-06T21:40:08.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 shutdown_connections 2023-07-06T21:40:08.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdeec06acb0 0x7fdeec06d160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c0cfd10 0x7fdf0c1e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0b1a80 0x7fdf0c0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 --2- 172.21.15.84:0/711226648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdf0c0b81c0 0x7fdf0c15b560 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 >> 172.21.15.84:0/711226648 conn(0x7fdf0c06b2b0 msgr2=0x7fdf0c0cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:08.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 shutdown_connections 2023-07-06T21:40:08.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.018+0000 7fdf109ff700 1 -- 172.21.15.84:0/711226648 wait complete. 2023-07-06T21:40:08.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.232+0000 7f943e9ff700 1 Processor -- start 2023-07-06T21:40:08.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.232+0000 7f943e9ff700 1 -- start start 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f943e9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f94380b81c0 0x7f94380cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f943e9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f94380ceda0 0x7f94380cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f943e9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94380d1830 con 0x7f94380b1a80 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f943e9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380cf330 con 0x7f94380b81c0 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f943e9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94380d1270 con 0x7f94380ceda0 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f942ffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f94380b1a80 0x7f943815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55976/0 (socket says 172.21.15.84:55976) 2023-07-06T21:40:08.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f942ffff700 1 -- 172.21.15.84:0/1547461658 learned_addr learned my addr 172.21.15.84:0/1547461658 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f9437fff700 1 --2- 172.21.15.84:0/1547461658 >> v2:172.21.15.84:3300/0 conn(0x7f94380b81c0 0x7f94380cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f943cf9c700 1 --2- 172.21.15.84:0/1547461658 >> v2:172.21.15.107:3300/0 conn(0x7f94380ceda0 0x7f94380cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f9437fff700 1 -- 172.21.15.84:0/1547461658 >> v2:172.21.15.107:3300/0 conn(0x7f94380ceda0 msgr2=0x7f94380cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f9437fff700 1 --2- 172.21.15.84:0/1547461658 >> v2:172.21.15.107:3300/0 conn(0x7f94380ceda0 0x7f94380cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f94377fe700 1 -- 172.21.15.84:0/1547461658 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808915133 0 0) 0x7f94380d1830 con 0x7f94380b1a80 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f9437fff700 1 -- 172.21.15.84:0/1547461658 >> v1:172.21.15.84:6789/0 conn(0x7f94380b1a80 legacy=0x7f943815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.234+0000 7f9437fff700 1 -- 172.21.15.84:0/1547461658 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380d1550 con 0x7f94380b81c0 2023-07-06T21:40:08.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.235+0000 7f9437fff700 1 --2- 172.21.15.84:0/1547461658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f94380cf600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94380c9c50 tx=0x7f9428009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e14858b92638f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.235+0000 7f94377fe700 1 -- 172.21.15.84:0/1547461658 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011830 con 0x7f94380b81c0 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.235+0000 7f94377fe700 1 -- 172.21.15.84:0/1547461658 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9428011990 con 0x7f94380b81c0 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.235+0000 7f94377fe700 1 -- 172.21.15.84:0/1547461658 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9428011d20 con 0x7f94380b81c0 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.235+0000 7f943e9ff700 1 -- 172.21.15.84:0/1547461658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 msgr2=0x7f94380cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.235+0000 7f943e9ff700 1 --2- 172.21.15.84:0/1547461658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f94380cf600 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94380c9c50 tx=0x7f9428009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.236+0000 7f943e9ff700 1 -- 172.21.15.84:0/1547461658 shutdown_connections 2023-07-06T21:40:08.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.236+0000 7f943e9ff700 1 --2- 172.21.15.84:0/1547461658 >> v2:172.21.15.107:3300/0 conn(0x7f94380ceda0 0x7f94380cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.236+0000 7f943e9ff700 1 --2- 172.21.15.84:0/1547461658 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f94380cf600 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.236+0000 7f943e9ff700 1 -- 172.21.15.84:0/1547461658 >> 172.21.15.84:0/1547461658 conn(0x7f943806b2b0 msgr2=0x7f943806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.236+0000 7f943e9ff700 1 -- 172.21.15.84:0/1547461658 shutdown_connections 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.236+0000 7f943e9ff700 1 -- 172.21.15.84:0/1547461658 wait complete. 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.237+0000 7f943e9ff700 1 Processor -- start 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.238+0000 7f943e9ff700 1 -- start start 2023-07-06T21:40:08.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.238+0000 7f943e9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94380b1a80 0x7f94381596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.238+0000 7f943e9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f9438156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.238+0000 7f943e9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94380ceda0 0x7f94380b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f943e9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b3570 con 0x7f94380b81c0 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f943e9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438159be0 con 0x7f94380b1a80 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f943e9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380b28f0 con 0x7f94380ceda0 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f9438156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f9438156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57266/0 (socket says 172.21.15.84:57266) 2023-07-06T21:40:08.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 -- 172.21.15.84:0/2456546037 learned_addr learned my addr 172.21.15.84:0/2456546037 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:08.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f9437fff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94380b1a80 0x7f94381596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f943cf9c700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94380ceda0 0x7f94380b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 -- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94380ceda0 msgr2=0x7f94380b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94380ceda0 0x7f94380b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 -- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94380b1a80 msgr2=0x7f94381596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94380b1a80 0x7f94381596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 -- 172.21.15.84:0/2456546037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94280094a0 con 0x7f94380b81c0 2023-07-06T21:40:08.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.239+0000 7f942ffff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f9438156b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f942000c550 tx=0x7f942000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:08.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.240+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420004c80 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.240+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9420004e00 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.240+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420007310 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.240+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380b2bd0 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.240+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94380b3190 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.241+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943804e1d0 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.242+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9420007470 con 0x7f94380b81c0 2023-07-06T21:40:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.243+0000 7f94357fa700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f941806ace0 0x7f941806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.243+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9420089130 con 0x7f94380b81c0 2023-07-06T21:40:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.243+0000 7f9437fff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f941806ace0 0x7f941806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.244+0000 7f9437fff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f941806ace0 0x7f941806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94280090b0 tx=0x7f9428008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.247+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9420054190 con 0x7f94380b81c0 2023-07-06T21:40:08.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.442+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94380b4f30 con 0x7f94380b81c0 2023-07-06T21:40:08.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.442+0000 7f94357fa700 1 -- 172.21.15.84:0/2456546037 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94200577b0 con 0x7f94380b81c0 2023-07-06T21:40:08.445 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.444+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f941806ace0 msgr2=0x7f941806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.444+0000 7f943e9ff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f941806ace0 0x7f941806d190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f94280090b0 tx=0x7f9428008030 comp rx=0 tx=0).stop 2023-07-06T21:40:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.444+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 msgr2=0x7f9438156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.444+0000 7f943e9ff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f9438156b70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f942000c550 tx=0x7f942000c860 comp rx=0 tx=0).stop 2023-07-06T21:40:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 shutdown_connections 2023-07-06T21:40:08.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f941806ace0 0x7f941806d190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94380ceda0 0x7f94380b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94380b1a80 0x7f94381596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 --2- 172.21.15.84:0/2456546037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94380b81c0 0x7f9438156b70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 >> 172.21.15.84:0/2456546037 conn(0x7f943806b2b0 msgr2=0x7f94380b71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:08.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 shutdown_connections 2023-07-06T21:40:08.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.445+0000 7f943e9ff700 1 -- 172.21.15.84:0/2456546037 wait complete. 2023-07-06T21:40:08.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:08.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.661+0000 7f6f033ff700 1 Processor -- start 2023-07-06T21:40:08.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.661+0000 7f6f033ff700 1 -- start start 2023-07-06T21:40:08.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.662+0000 7f6f033ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6efc0b1a80 0x7f6efc0d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.662+0000 7f6f033ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0d1830 con 0x7f6efc0cfd10 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.662+0000 7f6f033ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6efc0cf330 con 0x7f6efc0b81c0 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.662+0000 7f6f033ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6efc0d1270 con 0x7f6efc0b1a80 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.662+0000 7f6f0099a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6efc0cfd10 0x7f6efc0d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55992/0 (socket says 172.21.15.84:55992) 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6f0099a700 1 -- 172.21.15.84:0/1499663664 learned_addr learned my addr 172.21.15.84:0/1499663664 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6f0199c700 1 --2- 172.21.15.84:0/1499663664 >> v2:172.21.15.84:3300/0 conn(0x7f6efc0b1a80 0x7f6efc0d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6ef3fff700 1 -- 172.21.15.84:0/1499663664 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310873779 0 0) 0x7f6efc0d1830 con 0x7f6efc0cfd10 2023-07-06T21:40:08.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6ef3fff700 1 -- 172.21.15.84:0/1499663664 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee80035b0 con 0x7f6efc0cfd10 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6f0199c700 1 -- 172.21.15.84:0/1499663664 >> v1:172.21.15.107:6789/0 conn(0x7f6efc0b81c0 legacy=0x7f6efc0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6f0199c700 1 -- 172.21.15.84:0/1499663664 >> v1:172.21.15.84:6789/0 conn(0x7f6efc0cfd10 legacy=0x7f6efc0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6f0199c700 1 -- 172.21.15.84:0/1499663664 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc0d1550 con 0x7f6efc0b1a80 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6f0199c700 1 --2- 172.21.15.84:0/1499663664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0b1a80 0x7f6efc0d2a90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d90 tx=0x7f6ef400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97717a04d8403b55 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6ef3fff700 1 -- 172.21.15.84:0/1499663664 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef400e070 con 0x7f6efc0b1a80 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.663+0000 7f6ef3fff700 1 -- 172.21.15.84:0/1499663664 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef400bb20 con 0x7f6efc0b1a80 2023-07-06T21:40:08.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6ef3fff700 1 -- 172.21.15.84:0/1499663664 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4012430 con 0x7f6efc0b1a80 2023-07-06T21:40:08.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 -- 172.21.15.84:0/1499663664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0b1a80 msgr2=0x7f6efc0d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1499663664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0b1a80 0x7f6efc0d2a90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d90 tx=0x7f6ef400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:08.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 -- 172.21.15.84:0/1499663664 shutdown_connections 2023-07-06T21:40:08.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1499663664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0b1a80 0x7f6efc0d2a90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 -- 172.21.15.84:0/1499663664 >> 172.21.15.84:0/1499663664 conn(0x7f6efc06b2b0 msgr2=0x7f6efc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:08.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 -- 172.21.15.84:0/1499663664 shutdown_connections 2023-07-06T21:40:08.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.664+0000 7f6f033ff700 1 -- 172.21.15.84:0/1499663664 wait complete. 2023-07-06T21:40:08.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.666+0000 7f6f033ff700 1 Processor -- start 2023-07-06T21:40:08.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.666+0000 7f6f033ff700 1 -- start start 2023-07-06T21:40:08.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f033ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6efc0b1a80 0x7f6efc0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f033ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc0b81c0 0x7f6efc0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f033ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 0x7f6efc149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f033ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c79f0 con 0x7f6efc0cfd10 2023-07-06T21:40:08.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f033ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0cb360 con 0x7f6efc0b81c0 2023-07-06T21:40:08.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f033ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc0c6d70 con 0x7f6efc0b1a80 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f0199c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 0x7f6efc149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f0199c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 0x7f6efc149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57290/0 (socket says 172.21.15.84:57290) 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.667+0000 7f6f0199c700 1 -- 172.21.15.84:0/1261262055 learned_addr learned my addr 172.21.15.84:0/1261262055 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0099a700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc0b81c0 0x7f6efc0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0119b700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6efc0b1a80 0x7f6efc0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0199c700 1 -- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6efc0b1a80 msgr2=0x7f6efc0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0199c700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6efc0b1a80 0x7f6efc0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0199c700 1 -- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc0b81c0 msgr2=0x7f6efc0cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0199c700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc0b81c0 0x7f6efc0cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0199c700 1 -- 172.21.15.84:0/1261262055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef4009a70 con 0x7f6efc0cfd10 2023-07-06T21:40:08.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0099a700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc0b81c0 0x7f6efc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0119b700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6efc0b1a80 0x7f6efc0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f0199c700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 0x7f6efc149060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6efc06bed0 tx=0x7f6ef4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4003c20 con 0x7f6efc0cfd10 2023-07-06T21:40:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef4020070 con 0x7f6efc0cfd10 2023-07-06T21:40:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef401b610 con 0x7f6efc0cfd10 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc0c7050 con 0x7f6efc0cfd10 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.668+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc0c7610 con 0x7f6efc0cfd10 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.669+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc0c7b30 con 0x7f6efc0cfd10 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.670+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ef401b770 con 0x7f6efc0cfd10 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.671+0000 7f6ef1ffb700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6ee006ace0 0x7f6ee006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.672+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6ef4022070 con 0x7f6efc0cfd10 2023-07-06T21:40:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.672+0000 7f6f0119b700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6ee006ace0 0x7f6ee006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.672+0000 7f6f0119b700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6ee006ace0 0x7f6ee006d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000f60 tx=0x7f6ef8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:08.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.676+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6efc0c7b30 con 0x7f6efc0cfd10 2023-07-06T21:40:08.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.876+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6efc0c7b30 con 0x7f6efc0cfd10 2023-07-06T21:40:08.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.876+0000 7f6ef1ffb700 1 -- 172.21.15.84:0/1261262055 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6efc0c7b30 con 0x7f6efc0cfd10 2023-07-06T21:40:08.878 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:08.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.878+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6ee006ace0 msgr2=0x7f6ee006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.878+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6ee006ace0 0x7f6ee006d190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000f60 tx=0x7f6ef8009040 comp rx=0 tx=0).stop 2023-07-06T21:40:08.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.878+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 msgr2=0x7f6efc149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:08.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.878+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 0x7f6efc149060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6efc06bed0 tx=0x7f6ef4004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 shutdown_connections 2023-07-06T21:40:08.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6ee006ace0 0x7f6ee006d190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6efc0cfd10 0x7f6efc149060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6efc0b1a80 0x7f6efc0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 --2- 172.21.15.84:0/1261262055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6efc0b81c0 0x7f6efc0cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:08.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 >> 172.21.15.84:0/1261262055 conn(0x7f6efc06b2b0 msgr2=0x7f6efc0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:08.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 shutdown_connections 2023-07-06T21:40:08.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:08.879+0000 7f6f033ff700 1 -- 172.21.15.84:0/1261262055 wait complete. 2023-07-06T21:40:08.928 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-06T21:40:08.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-07-06T21:40:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.126+0000 7f87061ff700 1 Processor -- start 2023-07-06T21:40:09.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.126+0000 7f87061ff700 1 -- start start 2023-07-06T21:40:09.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f87061ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87000d1580 0x7f8700166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:09.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f87061ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000d0990 con 0x7f87000b8530 2023-07-06T21:40:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f87061ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87000cd9d0 con 0x7f87000d3580 2023-07-06T21:40:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f87061ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87000d3af0 con 0x7f87000d1580 2023-07-06T21:40:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f86ff7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f87000b8530 0x7f87000cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:55998/0 (socket says 172.21.15.84:55998) 2023-07-06T21:40:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f86fffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87000d1580 0x7f8700166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.128+0000 7f86ff7fe700 1 -- 172.21.15.84:0/4064666179 learned_addr learned my addr 172.21.15.84:0/4064666179 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fffff700 1 -- 172.21.15.84:0/4064666179 >> v1:172.21.15.107:6789/0 conn(0x7f87000d3580 legacy=0x7f87000d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fffff700 1 -- 172.21.15.84:0/4064666179 >> v1:172.21.15.84:6789/0 conn(0x7f87000b8530 legacy=0x7f87000cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fffff700 1 -- 172.21.15.84:0/4064666179 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87000cd410 con 0x7f87000d1580 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fe7fc700 1 -- 172.21.15.84:0/4064666179 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167658654 0 0) 0x7f87000d0990 con 0x7f87000b8530 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fffff700 1 --2- 172.21.15.84:0/4064666179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d1580 0x7f8700166ff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009d90 tx=0x7f86f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab341ed329063da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fe7fc700 1 -- 172.21.15.84:0/4064666179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f800e070 con 0x7f87000d1580 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fe7fc700 1 -- 172.21.15.84:0/4064666179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f800bb90 con 0x7f87000d1580 2023-07-06T21:40:09.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.129+0000 7f86fe7fc700 1 -- 172.21.15.84:0/4064666179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8003a20 con 0x7f87000d1580 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 -- 172.21.15.84:0/4064666179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d1580 msgr2=0x7f8700166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 --2- 172.21.15.84:0/4064666179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d1580 0x7f8700166ff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86f8009d90 tx=0x7f86f800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 -- 172.21.15.84:0/4064666179 shutdown_connections 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 --2- 172.21.15.84:0/4064666179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d1580 0x7f8700166ff0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 -- 172.21.15.84:0/4064666179 >> 172.21.15.84:0/4064666179 conn(0x7f87000715c0 msgr2=0x7f8700071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 -- 172.21.15.84:0/4064666179 shutdown_connections 2023-07-06T21:40:09.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.130+0000 7f87061ff700 1 -- 172.21.15.84:0/4064666179 wait complete. 2023-07-06T21:40:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.132+0000 7f87061ff700 1 Processor -- start 2023-07-06T21:40:09.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.132+0000 7f87061ff700 1 -- start start 2023-07-06T21:40:09.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f87061ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87000b8530 0x7f87001576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f87061ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87000d1580 0x7f8700151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f87061ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 0x7f87001f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f87061ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000c9cf0 con 0x7f87000d3580 2023-07-06T21:40:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f87061ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87000cb5e0 con 0x7f87000d1580 2023-07-06T21:40:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f87061ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87000c9f40 con 0x7f87000b8530 2023-07-06T21:40:09.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 0x7f87001f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 0x7f87001f60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57308/0 (socket says 172.21.15.84:57308) 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 -- 172.21.15.84:0/3144924881 learned_addr learned my addr 172.21.15.84:0/3144924881 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86ff7fe700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87000b8530 0x7f87001576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86feffd700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87000d1580 0x7f8700151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 -- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87000b8530 msgr2=0x7f87001576d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87000b8530 0x7f87001576d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 -- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87000d1580 msgr2=0x7f8700151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:09.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.134+0000 7f86fffff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87000d1580 0x7f8700151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f86fffff700 1 -- 172.21.15.84:0/3144924881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f8009a70 con 0x7f87000d3580 2023-07-06T21:40:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f86fffff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 0x7f87001f60e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b730 tx=0x7f86f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:09.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8012630 con 0x7f87000d3580 2023-07-06T21:40:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f80127b0 con 0x7f87000d3580 2023-07-06T21:40:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f8012b60 con 0x7f87000d3580 2023-07-06T21:40:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87000b2fd0 con 0x7f87000d3580 2023-07-06T21:40:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.135+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87000b3590 con 0x7f87000d3580 2023-07-06T21:40:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.136+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87000b3780 con 0x7f87000d3580 2023-07-06T21:40:09.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.137+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86f8012cc0 con 0x7f87000d3580 2023-07-06T21:40:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.138+0000 7f86e7fff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86ec06ace0 0x7f86ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.138+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f86f808b4e0 con 0x7f87000d3580 2023-07-06T21:40:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.138+0000 7f86ff7fe700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86ec06ace0 0x7f86ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:09.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.139+0000 7f86ff7fe700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86ec06ace0 0x7f86ec06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86f0007fd0 tx=0x7f86f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:09.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.143+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86f805a070 con 0x7f87000d3580 2023-07-06T21:40:09.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.417+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f8700065ea0 con 0x7f87000d3580 2023-07-06T21:40:09.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.990+0000 7f86e7fff700 1 -- 172.21.15.84:0/3144924881 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v65) v1 ==== 150+0+0 (secure 0 0 0) 0x7f86f8059c00 con 0x7f87000d3580 2023-07-06T21:40:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86ec06ace0 msgr2=0x7f86ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:09.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86ec06ace0 0x7f86ec06d190 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86f0007fd0 tx=0x7f86f0009040 comp rx=0 tx=0).stop 2023-07-06T21:40:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 msgr2=0x7f87001f60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 0x7f87001f60e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b730 tx=0x7f86f8004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 shutdown_connections 2023-07-06T21:40:09.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86ec06ace0 0x7f86ec06d190 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87000d3580 0x7f87001f60e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87000b8530 0x7f87001576d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 --2- 172.21.15.84:0/3144924881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87000d1580 0x7f8700151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:09.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 >> 172.21.15.84:0/3144924881 conn(0x7f87000715c0 msgr2=0x7f8700073430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:09.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 shutdown_connections 2023-07-06T21:40:09.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:09.992+0000 7f87061ff700 1 -- 172.21.15.84:0/3144924881 wait complete. 2023-07-06T21:40:10.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.205+0000 7fa826fff700 1 Processor -- start 2023-07-06T21:40:10.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.206+0000 7fa826fff700 1 -- start start 2023-07-06T21:40:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa826fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8200d3710 0x7fa8200cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa826fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8200cd910 con 0x7fa8200b83c0 2023-07-06T21:40:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa826fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8200d0a80 con 0x7fa8200cfa90 2023-07-06T21:40:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa826fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8200cff10 con 0x7fa8200d3710 2023-07-06T21:40:10.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa81ffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8200d3710 0x7fa8200cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa81ffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa8200d3710 0x7fa8200cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57312/0 (socket says 172.21.15.84:57312) 2023-07-06T21:40:10.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa81ffff700 1 -- 172.21.15.84:0/1750370526 learned_addr learned my addr 172.21.15.84:0/1750370526 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:10.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa824d9b700 1 --1- 172.21.15.84:0/1750370526 >> v1:172.21.15.84:6789/0 conn(0x7fa8200b83c0 0x7fa8200cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56000/0 (socket says 172.21.15.84:56000) 2023-07-06T21:40:10.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa81ffff700 1 -- 172.21.15.84:0/1750370526 >> v1:172.21.15.107:6789/0 conn(0x7fa8200cfa90 legacy=0x7fa8200d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:10.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa81ffff700 1 -- 172.21.15.84:0/1750370526 >> v1:172.21.15.84:6789/0 conn(0x7fa8200b83c0 legacy=0x7fa8200cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.207+0000 7fa81ffff700 1 -- 172.21.15.84:0/1750370526 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200b8790 con 0x7fa8200d3710 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa81ffff700 1 --2- 172.21.15.84:0/1750370526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa8200cf550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0098f0 tx=0x7fa80c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4937de74c0fd3ce0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa81f7fe700 1 -- 172.21.15.84:0/1750370526 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c004230 con 0x7fa8200d3710 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa81f7fe700 1 -- 172.21.15.84:0/1750370526 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c004390 con 0x7fa8200d3710 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa81f7fe700 1 -- 172.21.15.84:0/1750370526 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0036a0 con 0x7fa8200d3710 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa826fff700 1 -- 172.21.15.84:0/1750370526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 msgr2=0x7fa8200cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa826fff700 1 --2- 172.21.15.84:0/1750370526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa8200cf550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa80c0098f0 tx=0x7fa80c009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa826fff700 1 -- 172.21.15.84:0/1750370526 shutdown_connections 2023-07-06T21:40:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.208+0000 7fa826fff700 1 --2- 172.21.15.84:0/1750370526 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa8200cf550 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.209+0000 7fa826fff700 1 -- 172.21.15.84:0/1750370526 >> 172.21.15.84:0/1750370526 conn(0x7fa82006b350 msgr2=0x7fa82006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.209+0000 7fa826fff700 1 -- 172.21.15.84:0/1750370526 shutdown_connections 2023-07-06T21:40:10.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.209+0000 7fa826fff700 1 -- 172.21.15.84:0/1750370526 wait complete. 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.210+0000 7fa826fff700 1 Processor -- start 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.211+0000 7fa826fff700 1 -- start start 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.211+0000 7fa826fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8200b83c0 0x7fa8200cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.211+0000 7fa826fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8200cfa90 0x7fa8200b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa826fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa820150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa826fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200ca7e0 con 0x7fa8200d3710 2023-07-06T21:40:10.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa826fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200c9c10 con 0x7fa8200cfa90 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa826fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8200cb1b0 con 0x7fa8200b83c0 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa820150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa820150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57324/0 (socket says 172.21.15.84:57324) 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 -- 172.21.15.84:0/2980926557 learned_addr learned my addr 172.21.15.84:0/2980926557 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa81ffff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8200cfa90 0x7fa8200b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa824d9b700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8200b83c0 0x7fa8200cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 -- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8200b83c0 msgr2=0x7fa8200cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8200b83c0 0x7fa8200cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 -- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8200cfa90 msgr2=0x7fa8200b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8200cfa90 0x7fa8200b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 -- 172.21.15.84:0/2980926557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c009550 con 0x7fa8200d3710 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.212+0000 7fa82559c700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa820150ec0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b970 tx=0x7fa81800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.213+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800c930 con 0x7fa8200d3710 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.213+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81800cad0 con 0x7fa8200d3710 2023-07-06T21:40:10.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.213+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81800ce80 con 0x7fa8200d3710 2023-07-06T21:40:10.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.213+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8200c9eb0 con 0x7fa8200d3710 2023-07-06T21:40:10.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.213+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8200ca470 con 0x7fa8200d3710 2023-07-06T21:40:10.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.214+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8200ca950 con 0x7fa8200d3710 2023-07-06T21:40:10.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.215+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa818014440 con 0x7fa8200d3710 2023-07-06T21:40:10.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.216+0000 7fa81d7fa700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa80406ace0 0x7fa80406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.216+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa818088ff0 con 0x7fa8200d3710 2023-07-06T21:40:10.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.216+0000 7fa824d9b700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa80406ace0 0x7fa80406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.217+0000 7fa824d9b700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa80406ace0 0x7fa80406d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa814000f60 tx=0x7fa814007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:10.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.220+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa8180bb310 con 0x7fa8200d3710 2023-07-06T21:40:10.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.418+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa82015c110 con 0x7fa8200d3710 2023-07-06T21:40:10.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.419+0000 7fa81d7fa700 1 -- 172.21.15.84:0/2980926557 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7fa818019090 con 0x7fa8200d3710 2023-07-06T21:40:10.422 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:10.422 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa80406ace0 msgr2=0x7fa80406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa80406ace0 0x7fa80406d190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa814000f60 tx=0x7fa814007d70 comp rx=0 tx=0).stop 2023-07-06T21:40:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 msgr2=0x7fa820150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa820150ec0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa81800b970 tx=0x7fa81800bc80 comp rx=0 tx=0).stop 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 shutdown_connections 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa80406ace0 0x7fa80406d190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa8200d3710 0x7fa820150ec0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8200b83c0 0x7fa8200cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 --2- 172.21.15.84:0/2980926557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8200cfa90 0x7fa8200b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 >> 172.21.15.84:0/2980926557 conn(0x7fa82006b350 msgr2=0x7fa82015fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:10.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 shutdown_connections 2023-07-06T21:40:10.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.422+0000 7fa826fff700 1 -- 172.21.15.84:0/2980926557 wait complete. 2023-07-06T21:40:10.444 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:10.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:10.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.640+0000 7f36895ff700 1 Processor -- start 2023-07-06T21:40:10.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.640+0000 7f36895ff700 1 -- start start 2023-07-06T21:40:10.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.641+0000 7f36895ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f36840b81c0 0x7f36840cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.641+0000 7f36895ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.641+0000 7f36895ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36840d1830 con 0x7f36840ceda0 2023-07-06T21:40:10.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.641+0000 7f36895ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36840cf330 con 0x7f36840b81c0 2023-07-06T21:40:10.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.641+0000 7f36895ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36840d1270 con 0x7f36840b1a80 2023-07-06T21:40:10.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f36827fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f36837fe700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f36840ceda0 0x7f368415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:48742/0 (socket says 172.21.15.84:48742) 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3682ffd700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f36840b81c0 0x7f36840cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f36837fe700 1 -- 172.21.15.84:0/3342838460 learned_addr learned my addr 172.21.15.84:0/3342838460 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3681ffb700 1 -- 172.21.15.84:0/3342838460 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701948594 0 0) 0x7f36840d1830 con 0x7f36840ceda0 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3681ffb700 1 -- 172.21.15.84:0/3342838460 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366c0035b0 con 0x7f36840ceda0 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3682ffd700 1 -- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 msgr2=0x7f36840cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:10.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3682ffd700 1 --2- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3682ffd700 1 -- 172.21.15.84:0/3342838460 >> v1:172.21.15.107:6789/0 conn(0x7f36840ceda0 legacy=0x7f368415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f3682ffd700 1 -- 172.21.15.84:0/3342838460 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36840d1550 con 0x7f36840b81c0 2023-07-06T21:40:10.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.642+0000 7f36827fc700 1 --2- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:10.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f3682ffd700 1 --2- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840b81c0 0x7f36840cf600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3674009840 tx=0x7f3674009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7052258681f455 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:10.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f3681ffb700 1 -- 172.21.15.84:0/3342838460 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011830 con 0x7f36840b81c0 2023-07-06T21:40:10.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f3681ffb700 1 -- 172.21.15.84:0/3342838460 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674011990 con 0x7f36840b81c0 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f3681ffb700 1 -- 172.21.15.84:0/3342838460 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674011d20 con 0x7f36840b81c0 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 -- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840b81c0 msgr2=0x7f36840cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 --2- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840b81c0 0x7f36840cf600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3674009840 tx=0x7f3674009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 -- 172.21.15.84:0/3342838460 shutdown_connections 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 --2- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840b81c0 0x7f36840cf600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 --2- 172.21.15.84:0/3342838460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 -- 172.21.15.84:0/3342838460 >> 172.21.15.84:0/3342838460 conn(0x7f368406b2b0 msgr2=0x7f368406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:10.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 -- 172.21.15.84:0/3342838460 shutdown_connections 2023-07-06T21:40:10.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.643+0000 7f36895ff700 1 -- 172.21.15.84:0/3342838460 wait complete. 2023-07-06T21:40:10.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.645+0000 7f36895ff700 1 Processor -- start 2023-07-06T21:40:10.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.645+0000 7f36895ff700 1 -- start start 2023-07-06T21:40:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36895ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36895ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 0x7f36840cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36895ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840ceda0 0x7f36841f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36895ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36841f55f0 con 0x7f36840b81c0 2023-07-06T21:40:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36895ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36841f5890 con 0x7f36840ceda0 2023-07-06T21:40:10.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36895ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36841f5b70 con 0x7f36840b1a80 2023-07-06T21:40:10.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36827fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 0x7f36840cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36827fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 0x7f36840cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57326/0 (socket says 172.21.15.84:57326) 2023-07-06T21:40:10.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f36827fc700 1 -- 172.21.15.84:0/601658051 learned_addr learned my addr 172.21.15.84:0/601658051 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:10.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.646+0000 7f3682ffd700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36837fe700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840ceda0 0x7f36841f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36827fc700 1 -- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 msgr2=0x7f36840c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36827fc700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36827fc700 1 -- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840ceda0 msgr2=0x7f36841f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36827fc700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840ceda0 0x7f36841f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36827fc700 1 -- 172.21.15.84:0/601658051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36780094a0 con 0x7f36840b81c0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36827fc700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 0x7f36840cc100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f367800ebe0 tx=0x7f367800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3678009800 con 0x7f36840b81c0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367800c910 con 0x7f36840b81c0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367800cc80 con 0x7f36840b81c0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36740094a0 con 0x7f36840b81c0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.647+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36841f6130 con 0x7f36840b81c0 2023-07-06T21:40:10.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.648+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36841f62f0 con 0x7f36840b81c0 2023-07-06T21:40:10.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.649+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3678013470 con 0x7f36840b81c0 2023-07-06T21:40:10.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.650+0000 7f366b7fe700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f367006ace0 0x7f367006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:10.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.650+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f36780896c0 con 0x7f36840b81c0 2023-07-06T21:40:10.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.650+0000 7f3682ffd700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f367006ace0 0x7f367006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:10.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.651+0000 7f3682ffd700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f367006ace0 0x7f367006d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:10.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.654+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3678058120 con 0x7f36840b81c0 2023-07-06T21:40:10.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.849+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f368404e1d0 con 0x7f36840b81c0 2023-07-06T21:40:10.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.849+0000 7f366b7fe700 1 -- 172.21.15.84:0/601658051 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3678057cb0 con 0x7f36840b81c0 2023-07-06T21:40:10.852 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.851+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f367006ace0 msgr2=0x7f367006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.851+0000 7f36895ff700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f367006ace0 0x7f367006d190 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674008030 comp rx=0 tx=0).stop 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.851+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 msgr2=0x7f36840cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.851+0000 7f36895ff700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 0x7f36840cc100 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f367800ebe0 tx=0x7f367800eef0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 shutdown_connections 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f367006ace0 0x7f367006d190 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36840ceda0 0x7f36841f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36840b1a80 0x7f36840c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 --2- 172.21.15.84:0/601658051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36840b81c0 0x7f36840cc100 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:10.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 >> 172.21.15.84:0/601658051 conn(0x7f368406b2b0 msgr2=0x7f36840b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:10.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 shutdown_connections 2023-07-06T21:40:10.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:10.852+0000 7f36895ff700 1 -- 172.21.15.84:0/601658051 wait complete. 2023-07-06T21:40:10.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:11.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.066+0000 7f096c3ff700 1 Processor -- start 2023-07-06T21:40:11.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.066+0000 7f096c3ff700 1 -- start start 2023-07-06T21:40:11.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.067+0000 7f096c3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09640cfd10 0x7f09640d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.067+0000 7f096c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09640d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096c3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09640d1830 con 0x7f09640b81c0 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096c3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09640cf330 con 0x7f09640cfd10 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09640d1270 con 0x7f09640b1a80 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09640cfd10 0x7f09640d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f09640cfd10 0x7f09640d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57332/0 (socket says 172.21.15.84:57332) 2023-07-06T21:40:11.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 -- 172.21.15.84:0/3340062439 learned_addr learned my addr 172.21.15.84:0/3340062439 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:11.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096a99c700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09640d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 -- 172.21.15.84:0/3340062439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 msgr2=0x7f09640d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:11.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09640d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 -- 172.21.15.84:0/3340062439 >> v1:172.21.15.107:6789/0 conn(0x7f09640b81c0 legacy=0x7f09640cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:11.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096999a700 1 -- 172.21.15.84:0/3340062439 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09640d1550 con 0x7f09640cfd10 2023-07-06T21:40:11.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.068+0000 7f096a99c700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09640d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:11.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096999a700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640cfd10 0x7f09640d2380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f09540098f0 tx=0x7f0954009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c8c3f2b651744e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f0969199700 1 -- 172.21.15.84:0/3340062439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954004230 con 0x7f09640cfd10 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f0969199700 1 -- 172.21.15.84:0/3340062439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0954004390 con 0x7f09640cfd10 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f0969199700 1 -- 172.21.15.84:0/3340062439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09540036a0 con 0x7f09640cfd10 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 -- 172.21.15.84:0/3340062439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640cfd10 msgr2=0x7f09640d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640cfd10 0x7f09640d2380 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f09540098f0 tx=0x7f0954009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 -- 172.21.15.84:0/3340062439 shutdown_connections 2023-07-06T21:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09640d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 --2- 172.21.15.84:0/3340062439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640cfd10 0x7f09640d2380 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 -- 172.21.15.84:0/3340062439 >> 172.21.15.84:0/3340062439 conn(0x7f096406b2b0 msgr2=0x7f096406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:11.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.069+0000 7f096c3ff700 1 -- 172.21.15.84:0/3340062439 shutdown_connections 2023-07-06T21:40:11.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.070+0000 7f096c3ff700 1 -- 172.21.15.84:0/3340062439 wait complete. 2023-07-06T21:40:11.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.071+0000 7f096c3ff700 1 Processor -- start 2023-07-06T21:40:11.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.071+0000 7f096c3ff700 1 -- start start 2023-07-06T21:40:11.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.072+0000 7f096c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09641f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.072+0000 7f096c3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 0x7f0964152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.072+0000 7f096c3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09640cfd10 0x7f0964152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.072+0000 7f096c3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09641583e0 con 0x7f09640b81c0 2023-07-06T21:40:11.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.072+0000 7f096c3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09641f24c0 con 0x7f09640cfd10 2023-07-06T21:40:11.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.072+0000 7f096c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096415a1f0 con 0x7f09640b1a80 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 0x7f0964152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096a19b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09641f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 0x7f0964152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57336/0 (socket says 172.21.15.84:57336) 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 -- 172.21.15.84:0/81698191 learned_addr learned my addr 172.21.15.84:0/81698191 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096a99c700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09640cfd10 0x7f0964152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 -- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 msgr2=0x7f09641f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09641f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 -- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09640cfd10 msgr2=0x7f0964152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09640cfd10 0x7f0964152a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 -- 172.21.15.84:0/81698191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954009550 con 0x7f09640b81c0 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096a99c700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09640cfd10 0x7f0964152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096999a700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 0x7f0964152210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0954004000 tx=0x7f0954004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400b660 con 0x7f09640b81c0 2023-07-06T21:40:11.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095400b7e0 con 0x7f09640b81c0 2023-07-06T21:40:11.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09641588d0 con 0x7f09640b81c0 2023-07-06T21:40:11.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.073+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400bb90 con 0x7f09640b81c0 2023-07-06T21:40:11.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.074+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096415e4a0 con 0x7f09640b81c0 2023-07-06T21:40:11.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.074+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096404e1d0 con 0x7f09640b81c0 2023-07-06T21:40:11.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.076+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f095400bcf0 con 0x7f09640b81c0 2023-07-06T21:40:11.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.076+0000 7f095affd700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f094806ace0 0x7f094806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.077+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f095408ac50 con 0x7f09640b81c0 2023-07-06T21:40:11.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.077+0000 7f096a19b700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f094806ace0 0x7f094806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.077+0000 7f096a19b700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f094806ace0 0x7f094806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09641e8e30 tx=0x7f0960007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.081+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0954055cb0 con 0x7f09640b81c0 2023-07-06T21:40:11.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.276+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09640b4f30 con 0x7f09640b81c0 2023-07-06T21:40:11.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.276+0000 7f095affd700 1 -- 172.21.15.84:0/81698191 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f09540592d0 con 0x7f09640b81c0 2023-07-06T21:40:11.279 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.278+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f094806ace0 msgr2=0x7f094806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.278+0000 7f096c3ff700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f094806ace0 0x7f094806d190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f09641e8e30 tx=0x7f0960007ec0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.278+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 msgr2=0x7f0964152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.278+0000 7f096c3ff700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 0x7f0964152210 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0954004000 tx=0x7f0954004750 comp rx=0 tx=0).stop 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 shutdown_connections 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f094806ace0 0x7f094806d190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09640cfd10 0x7f0964152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09640b1a80 0x7f09641f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 --2- 172.21.15.84:0/81698191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09640b81c0 0x7f0964152210 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 >> 172.21.15.84:0/81698191 conn(0x7f096406b2b0 msgr2=0x7f09640c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:11.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 shutdown_connections 2023-07-06T21:40:11.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.279+0000 7f096c3ff700 1 -- 172.21.15.84:0/81698191 wait complete. 2023-07-06T21:40:11.299 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:11.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.492+0000 7fb5263ff700 1 Processor -- start 2023-07-06T21:40:11.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.492+0000 7fb5263ff700 1 -- start start 2023-07-06T21:40:11.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb5263ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb5200b81c0 0x7fb5200cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb5263ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5200ceda0 0x7fb5200cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb5263ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5200d1830 con 0x7fb5200b1a80 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb5263ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200cf330 con 0x7fb5200ceda0 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb5263ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5200d1270 con 0x7fb5200b81c0 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb52499c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5200ceda0 0x7fb5200cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb51f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb5200b1a80 0x7fb52015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56012/0 (socket says 172.21.15.84:56012) 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb51f7fe700 1 -- 172.21.15.84:0/1891953130 learned_addr learned my addr 172.21.15.84:0/1891953130 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb52499c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5200ceda0 0x7fb5200cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57338/0 (socket says 172.21.15.84:57338) 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb51ffff700 1 --2- 172.21.15.84:0/1891953130 >> v2:172.21.15.107:3300/0 conn(0x7fb5200b81c0 0x7fb5200cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb52499c700 1 -- 172.21.15.84:0/1891953130 >> v2:172.21.15.107:3300/0 conn(0x7fb5200b81c0 msgr2=0x7fb5200cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:11.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb52499c700 1 --2- 172.21.15.84:0/1891953130 >> v2:172.21.15.107:3300/0 conn(0x7fb5200b81c0 0x7fb5200cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb52499c700 1 -- 172.21.15.84:0/1891953130 >> v1:172.21.15.84:6789/0 conn(0x7fb5200b1a80 legacy=0x7fb52015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb52499c700 1 -- 172.21.15.84:0/1891953130 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200d1550 con 0x7fb5200ceda0 2023-07-06T21:40:11.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.494+0000 7fb51effd700 1 -- 172.21.15.84:0/1891953130 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899161658 0 0) 0x7fb5200d1830 con 0x7fb5200b1a80 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb52499c700 1 --2- 172.21.15.84:0/1891953130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200ceda0 0x7fb5200cfb40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb5200b2620 tx=0x7fb51800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf6315f906fcaae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb51effd700 1 -- 172.21.15.84:0/1891953130 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51800e070 con 0x7fb5200ceda0 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb51effd700 1 -- 172.21.15.84:0/1891953130 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51800bbd0 con 0x7fb5200ceda0 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb51effd700 1 -- 172.21.15.84:0/1891953130 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518012430 con 0x7fb5200ceda0 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb5263ff700 1 -- 172.21.15.84:0/1891953130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200ceda0 msgr2=0x7fb5200cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb5263ff700 1 --2- 172.21.15.84:0/1891953130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200ceda0 0x7fb5200cfb40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb5200b2620 tx=0x7fb51800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:11.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb5263ff700 1 -- 172.21.15.84:0/1891953130 shutdown_connections 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb5263ff700 1 --2- 172.21.15.84:0/1891953130 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200ceda0 0x7fb5200cfb40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb5263ff700 1 --2- 172.21.15.84:0/1891953130 >> v2:172.21.15.107:3300/0 conn(0x7fb5200b81c0 0x7fb5200cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.495+0000 7fb5263ff700 1 -- 172.21.15.84:0/1891953130 >> 172.21.15.84:0/1891953130 conn(0x7fb52006b2b0 msgr2=0x7fb52006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.496+0000 7fb5263ff700 1 -- 172.21.15.84:0/1891953130 shutdown_connections 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.496+0000 7fb5263ff700 1 -- 172.21.15.84:0/1891953130 wait complete. 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.497+0000 7fb5263ff700 1 Processor -- start 2023-07-06T21:40:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.497+0000 7fb5263ff700 1 -- start start 2023-07-06T21:40:11.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.498+0000 7fb5263ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 0x7fb5201f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.498+0000 7fb5263ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5200b81c0 0x7fb520157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.498+0000 7fb5263ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5200ceda0 0x7fb5201f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.498+0000 7fb5263ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201f2f00 con 0x7fb5200b1a80 2023-07-06T21:40:11.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.498+0000 7fb5263ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5201f6470 con 0x7fb5200b81c0 2023-07-06T21:40:11.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.498+0000 7fb5263ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb520157fe0 con 0x7fb5200ceda0 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 0x7fb5201f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 0x7fb5201f29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57340/0 (socket says 172.21.15.84:57340) 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 -- 172.21.15.84:0/2372401689 learned_addr learned my addr 172.21.15.84:0/2372401689 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51f7fe700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5200b81c0 0x7fb520157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 -- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5200ceda0 msgr2=0x7fb5201f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5200ceda0 0x7fb5201f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 -- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5200b81c0 msgr2=0x7fb520157aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5200b81c0 0x7fb520157aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 -- 172.21.15.84:0/2372401689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb518009a70 con 0x7fb5200b1a80 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51f7fe700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5200b81c0 0x7fb520157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51ffff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 0x7fb5201f29c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e810 tx=0x7fb51000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.499+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510013610 con 0x7fb5200b1a80 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.500+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb510013770 con 0x7fb5200b1a80 2023-07-06T21:40:11.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.500+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb510013b20 con 0x7fb5200b1a80 2023-07-06T21:40:11.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.500+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5201f58f0 con 0x7fb5200b1a80 2023-07-06T21:40:11.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.500+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5201f5eb0 con 0x7fb5200b1a80 2023-07-06T21:40:11.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.501+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52004e1d0 con 0x7fb5200b1a80 2023-07-06T21:40:11.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.502+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb510013c80 con 0x7fb5200b1a80 2023-07-06T21:40:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.503+0000 7fb51cff9700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb50c06ace0 0x7fb50c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.503+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb510089210 con 0x7fb5200b1a80 2023-07-06T21:40:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.503+0000 7fb51f7fe700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb50c06ace0 0x7fb50c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.503+0000 7fb51f7fe700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb50c06ace0 0x7fb50c06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb514007fd0 tx=0x7fb514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.506+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5100541f0 con 0x7fb5200b1a80 2023-07-06T21:40:11.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.704+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5201e5240 con 0x7fb5200b1a80 2023-07-06T21:40:11.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.705+0000 7fb51cff9700 1 -- 172.21.15.84:0/2372401689 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb510057810 con 0x7fb5200b1a80 2023-07-06T21:40:11.707 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:11.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb50c06ace0 msgr2=0x7fb50c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb50c06ace0 0x7fb50c06d190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb514007fd0 tx=0x7fb514009040 comp rx=0 tx=0).stop 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 msgr2=0x7fb5201f29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 0x7fb5201f29c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb51000e810 tx=0x7fb51000ebd0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 shutdown_connections 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb50c06ace0 0x7fb50c06d190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5200ceda0 0x7fb5201f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5200b1a80 0x7fb5201f29c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 --2- 172.21.15.84:0/2372401689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5200b81c0 0x7fb520157aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 >> 172.21.15.84:0/2372401689 conn(0x7fb52006b2b0 msgr2=0x7fb52015b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:11.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 shutdown_connections 2023-07-06T21:40:11.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.707+0000 7fb5263ff700 1 -- 172.21.15.84:0/2372401689 wait complete. 2023-07-06T21:40:11.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:11.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.923+0000 7f46a47ff700 1 Processor -- start 2023-07-06T21:40:11.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.923+0000 7f46a47ff700 1 -- start start 2023-07-06T21:40:11.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.924+0000 7f46a47ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f469c0cfd10 0x7f469c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.924+0000 7f46a47ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f469c0b1a80 0x7f469c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a47ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c0d1830 con 0x7f469c0b81c0 2023-07-06T21:40:11.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a47ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c0cf330 con 0x7f469c0cfd10 2023-07-06T21:40:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a47ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c0d1270 con 0x7f469c0b1a80 2023-07-06T21:40:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a259b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f469c0b81c0 0x7f469c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56028/0 (socket says 172.21.15.84:56028) 2023-07-06T21:40:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a259b700 1 -- 172.21.15.84:0/47037656 learned_addr learned my addr 172.21.15.84:0/47037656 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:11.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/47037656 >> v2:172.21.15.84:3300/0 conn(0x7f469c0cfd10 0x7f469c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a2d9c700 1 --2- 172.21.15.84:0/47037656 >> v2:172.21.15.107:3300/0 conn(0x7f469c0b1a80 0x7f469c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1599700 1 -- 172.21.15.84:0/47037656 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403472842 0 0) 0x7f469c0d1830 con 0x7f469c0b81c0 2023-07-06T21:40:11.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1599700 1 -- 172.21.15.84:0/47037656 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46880035b0 con 0x7f469c0b81c0 2023-07-06T21:40:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1d9a700 1 -- 172.21.15.84:0/47037656 >> v2:172.21.15.107:3300/0 conn(0x7f469c0b1a80 msgr2=0x7f469c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/47037656 >> v2:172.21.15.107:3300/0 conn(0x7f469c0b1a80 0x7f469c0d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1d9a700 1 -- 172.21.15.84:0/47037656 >> v1:172.21.15.84:6789/0 conn(0x7f469c0b81c0 legacy=0x7f469c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a1d9a700 1 -- 172.21.15.84:0/47037656 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0d1550 con 0x7f469c0cfd10 2023-07-06T21:40:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.925+0000 7f46a2d9c700 1 --2- 172.21.15.84:0/47037656 >> v2:172.21.15.107:3300/0 conn(0x7f469c0b1a80 0x7f469c0d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:11.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/47037656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0cfd10 0x7f469c0d2380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f469c0b8d30 tx=0x7f468c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c891c1bb10480f47 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a1599700 1 -- 172.21.15.84:0/47037656 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c011830 con 0x7f469c0cfd10 2023-07-06T21:40:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a1599700 1 -- 172.21.15.84:0/47037656 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c011990 con 0x7f469c0cfd10 2023-07-06T21:40:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a1599700 1 -- 172.21.15.84:0/47037656 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c011d00 con 0x7f469c0cfd10 2023-07-06T21:40:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 -- 172.21.15.84:0/47037656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0cfd10 msgr2=0x7f469c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 --2- 172.21.15.84:0/47037656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0cfd10 0x7f469c0d2380 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f469c0b8d30 tx=0x7f468c009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 -- 172.21.15.84:0/47037656 shutdown_connections 2023-07-06T21:40:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 --2- 172.21.15.84:0/47037656 >> v2:172.21.15.107:3300/0 conn(0x7f469c0b1a80 0x7f469c0d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 --2- 172.21.15.84:0/47037656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0cfd10 0x7f469c0d2380 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 -- 172.21.15.84:0/47037656 >> 172.21.15.84:0/47037656 conn(0x7f469c06b2b0 msgr2=0x7f469c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 -- 172.21.15.84:0/47037656 shutdown_connections 2023-07-06T21:40:11.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.926+0000 7f46a47ff700 1 -- 172.21.15.84:0/47037656 wait complete. 2023-07-06T21:40:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.929+0000 7f46a47ff700 1 Processor -- start 2023-07-06T21:40:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.929+0000 7f46a47ff700 1 -- start start 2023-07-06T21:40:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a47ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 0x7f469c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a47ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f469c0b81c0 0x7f469c0c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.938 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a47ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f469c0cfd10 0x7f469c14f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a47ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c1504b0 con 0x7f469c0b1a80 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a47ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0cba10 con 0x7f469c0cfd10 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a47ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0cbcf0 con 0x7f469c0b81c0 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.930+0000 7f46a259b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 0x7f469c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 0x7f469c0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57354/0 (socket says 172.21.15.84:57354) 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 -- 172.21.15.84:0/3138705876 learned_addr learned my addr 172.21.15.84:0/3138705876 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:11.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f469c0b81c0 0x7f469c0c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 -- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f469c0b81c0 msgr2=0x7f469c0c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f469c0b81c0 0x7f469c0c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 -- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f469c0cfd10 msgr2=0x7f469c14f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a2d9c700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f469c0cfd10 0x7f469c14f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f469c0cfd10 0x7f469c14f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 -- 172.21.15.84:0/3138705876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0094a0 con 0x7f469c0b1a80 2023-07-06T21:40:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f469c0b81c0 0x7f469c0c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a259b700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 0x7f469c0cae20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f469800c4d0 tx=0x7f469800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698005d70 con 0x7f469c0b1a80 2023-07-06T21:40:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46980070d0 con 0x7f469c0b1a80 2023-07-06T21:40:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698007440 con 0x7f469c0b1a80 2023-07-06T21:40:11.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.931+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c0cbfd0 con 0x7f469c0b1a80 2023-07-06T21:40:11.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.932+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c0cc590 con 0x7f469c0b1a80 2023-07-06T21:40:11.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.933+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c04e1d0 con 0x7f469c0b1a80 2023-07-06T21:40:11.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.933+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f46980075a0 con 0x7f469c0b1a80 2023-07-06T21:40:11.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.934+0000 7f4692ffd700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f468006ace0 0x7f468006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:11.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.935+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f46980889b0 con 0x7f469c0b1a80 2023-07-06T21:40:11.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.935+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f468006ace0 0x7f468006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:11.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.935+0000 7f46a1d9a700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f468006ace0 0x7f468006d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f469c0d0100 tx=0x7f468c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:11.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:11.938+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4698053a30 con 0x7f469c0b1a80 2023-07-06T21:40:12.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.137+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f469c0b4f30 con 0x7f469c0b1a80 2023-07-06T21:40:12.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.137+0000 7f4692ffd700 1 -- 172.21.15.84:0/3138705876 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46980170a0 con 0x7f469c0b1a80 2023-07-06T21:40:12.139 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f468006ace0 msgr2=0x7f468006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f468006ace0 0x7f468006d190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f469c0d0100 tx=0x7f468c008030 comp rx=0 tx=0).stop 2023-07-06T21:40:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 msgr2=0x7f469c0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 0x7f469c0cae20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f469800c4d0 tx=0x7f469800c890 comp rx=0 tx=0).stop 2023-07-06T21:40:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 shutdown_connections 2023-07-06T21:40:12.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f468006ace0 0x7f468006d190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f469c0cfd10 0x7f469c14f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f469c0b1a80 0x7f469c0cae20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 --2- 172.21.15.84:0/3138705876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f469c0b81c0 0x7f469c0c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.139+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 >> 172.21.15.84:0/3138705876 conn(0x7f469c06b2b0 msgr2=0x7f469c0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.140+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 shutdown_connections 2023-07-06T21:40:12.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.140+0000 7f46a47ff700 1 -- 172.21.15.84:0/3138705876 wait complete. 2023-07-06T21:40:12.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T21:40:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.354+0000 7fec253ff700 1 Processor -- start 2023-07-06T21:40:12.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.354+0000 7fec253ff700 1 -- start start 2023-07-06T21:40:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec253ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec200c90a0 0x7fec20069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:12.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec253ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec2006a3d0 0x7fec2006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec253ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2006cf40 con 0x7fec2015c020 2023-07-06T21:40:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec253ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2015d400 con 0x7fec2006a3d0 2023-07-06T21:40:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec253ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2006a7f0 con 0x7fec200c90a0 2023-07-06T21:40:12.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec1f7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec2006a3d0 0x7fec2006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:12.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec1effd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fec2015c020 0x7fec2015c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56032/0 (socket says 172.21.15.84:56032) 2023-07-06T21:40:12.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.355+0000 7fec1effd700 1 -- 172.21.15.84:0/3267402054 learned_addr learned my addr 172.21.15.84:0/3267402054 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1f7fe700 1 -- 172.21.15.84:0/3267402054 >> v2:172.21.15.107:3300/0 conn(0x7fec200c90a0 msgr2=0x7fec20069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1f7fe700 1 --2- 172.21.15.84:0/3267402054 >> v2:172.21.15.107:3300/0 conn(0x7fec200c90a0 0x7fec20069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1f7fe700 1 -- 172.21.15.84:0/3267402054 >> v1:172.21.15.84:6789/0 conn(0x7fec2015c020 legacy=0x7fec2015c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1f7fe700 1 -- 172.21.15.84:0/3267402054 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2015cb70 con 0x7fec2006a3d0 2023-07-06T21:40:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1dffb700 1 -- 172.21.15.84:0/3267402054 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697886595 0 0) 0x7fec2006cf40 con 0x7fec2015c020 2023-07-06T21:40:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1f7fe700 1 --2- 172.21.15.84:0/3267402054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2006a3d0 0x7fec2006d3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec200cc480 tx=0x7fec1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66eab625db0f8926 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1dffb700 1 -- 172.21.15.84:0/3267402054 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1800e070 con 0x7fec2006a3d0 2023-07-06T21:40:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1dffb700 1 -- 172.21.15.84:0/3267402054 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1800bbd0 con 0x7fec2006a3d0 2023-07-06T21:40:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.356+0000 7fec1dffb700 1 -- 172.21.15.84:0/3267402054 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec18012430 con 0x7fec2006a3d0 2023-07-06T21:40:12.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 -- 172.21.15.84:0/3267402054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2006a3d0 msgr2=0x7fec2006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 --2- 172.21.15.84:0/3267402054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2006a3d0 0x7fec2006d3d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec200cc480 tx=0x7fec1800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:12.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 -- 172.21.15.84:0/3267402054 shutdown_connections 2023-07-06T21:40:12.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 --2- 172.21.15.84:0/3267402054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2006a3d0 0x7fec2006d3d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 --2- 172.21.15.84:0/3267402054 >> v2:172.21.15.107:3300/0 conn(0x7fec200c90a0 0x7fec20069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 -- 172.21.15.84:0/3267402054 >> 172.21.15.84:0/3267402054 conn(0x7fec200cbf10 msgr2=0x7fec200ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:12.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 -- 172.21.15.84:0/3267402054 shutdown_connections 2023-07-06T21:40:12.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.357+0000 7fec253ff700 1 -- 172.21.15.84:0/3267402054 wait complete. 2023-07-06T21:40:12.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.359+0000 7fec253ff700 1 Processor -- start 2023-07-06T21:40:12.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.359+0000 7fec253ff700 1 -- start start 2023-07-06T21:40:12.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.359+0000 7fec253ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2006a3d0 0x7fec201f2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:12.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec253ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 0x7fec200b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:12.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec253ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2015c020 0x7fec200ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:12.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec253ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec201f27b0 con 0x7fec200c90a0 2023-07-06T21:40:12.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec253ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec20158720 con 0x7fec2006a3d0 2023-07-06T21:40:12.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec253ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec20158a00 con 0x7fec2015c020 2023-07-06T21:40:12.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 0x7fec200b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:12.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 0x7fec200b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57376/0 (socket says 172.21.15.84:57376) 2023-07-06T21:40:12.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 -- 172.21.15.84:0/1425281045 learned_addr learned my addr 172.21.15.84:0/1425281045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1effd700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2006a3d0 0x7fec201f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1f7fe700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2015c020 0x7fec200ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 -- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2015c020 msgr2=0x7fec200ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2015c020 0x7fec200ae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 -- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2006a3d0 msgr2=0x7fec201f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2006a3d0 0x7fec201f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 -- 172.21.15.84:0/1425281045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18009a70 con 0x7fec200c90a0 2023-07-06T21:40:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.360+0000 7fec1e7fc700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 0x7fec200b3470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec1400b4a0 tx=0x7fec1400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.361+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14011830 con 0x7fec200c90a0 2023-07-06T21:40:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.361+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec14011990 con 0x7fec200c90a0 2023-07-06T21:40:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.361+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec20158ce0 con 0x7fec200c90a0 2023-07-06T21:40:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.361+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14011d40 con 0x7fec200c90a0 2023-07-06T21:40:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.361+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec201592a0 con 0x7fec200c90a0 2023-07-06T21:40:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.362+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec2004e1d0 con 0x7fec200c90a0 2023-07-06T21:40:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.363+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec1401a400 con 0x7fec200c90a0 2023-07-06T21:40:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.363+0000 7fec077fe700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0c06ace0 0x7fec0c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:12.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.364+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fec14089560 con 0x7fec200c90a0 2023-07-06T21:40:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.364+0000 7fec1effd700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0c06ace0 0x7fec0c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:12.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.364+0000 7fec1effd700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0c06ace0 0x7fec0c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fec10000f60 tx=0x7fec10007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:12.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.368+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec14054540 con 0x7fec200c90a0 2023-07-06T21:40:12.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.566+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fec200b4780 con 0x7fec200c90a0 2023-07-06T21:40:12.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.570+0000 7fec077fe700 1 -- 172.21.15.84:0/1425281045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fec14057b60 con 0x7fec200c90a0 2023-07-06T21:40:12.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0c06ace0 msgr2=0x7fec0c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0c06ace0 0x7fec0c06d190 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fec10000f60 tx=0x7fec10007be0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 msgr2=0x7fec200b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 0x7fec200b3470 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec1400b4a0 tx=0x7fec1400b860 comp rx=0 tx=0).stop 2023-07-06T21:40:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 shutdown_connections 2023-07-06T21:40:12.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0c06ace0 0x7fec0c06d190 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2015c020 0x7fec200ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec2006a3d0 0x7fec201f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 --2- 172.21.15.84:0/1425281045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec200c90a0 0x7fec200b3470 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 >> 172.21.15.84:0/1425281045 conn(0x7fec200cbf10 msgr2=0x7fec200cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.572+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 shutdown_connections 2023-07-06T21:40:12.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:12.573+0000 7fec253ff700 1 -- 172.21.15.84:0/1425281045 wait complete. 2023-07-06T21:40:12.592 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T21:40:12.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:40:12.691 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:40:12.801 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T21:40:12.801 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:40:12.802 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:12.844 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:40:12.845 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:12.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:13.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.164+0000 7f51dabff700 1 Processor -- start 2023-07-06T21:40:13.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.164+0000 7f51dabff700 1 -- start start 2023-07-06T21:40:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.165+0000 7f51dabff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f51d4154910 0x7f51d4070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.165+0000 7f51dabff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51d415a7e0 0x7f51d415ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.165+0000 7f51dabff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d4070230 con 0x7f51d40c8d00 2023-07-06T21:40:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.165+0000 7f51dabff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d415b2a0 con 0x7f51d415a7e0 2023-07-06T21:40:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.165+0000 7f51dabff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d415b580 con 0x7f51d4154910 2023-07-06T21:40:13.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f51d415a7e0 0x7f51d415ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d919c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f51d40c8d00 0x7f51d40c9120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36264/0 (socket says 172.21.15.84:36264) 2023-07-06T21:40:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d919c700 1 -- 172.21.15.84:0/924242157 learned_addr learned my addr 172.21.15.84:0/924242157 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:13.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d899b700 1 --2- 172.21.15.84:0/924242157 >> v2:172.21.15.107:3300/0 conn(0x7f51d4154910 0x7f51d4070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d3fff700 1 -- 172.21.15.84:0/924242157 >> v2:172.21.15.107:3300/0 conn(0x7f51d4154910 msgr2=0x7f51d4070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d3fff700 1 --2- 172.21.15.84:0/924242157 >> v2:172.21.15.107:3300/0 conn(0x7f51d4154910 0x7f51d4070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d3fff700 1 -- 172.21.15.84:0/924242157 >> v1:172.21.15.84:6789/0 conn(0x7f51d40c8d00 legacy=0x7f51d40c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d3fff700 1 -- 172.21.15.84:0/924242157 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4070480 con 0x7f51d415a7e0 2023-07-06T21:40:13.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.166+0000 7f51d3fff700 1 --2- 172.21.15.84:0/924242157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d415a7e0 0x7f51d415ac30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51d40ce6a0 tx=0x7f51c0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd70669146fcea11 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51d37fe700 1 -- 172.21.15.84:0/924242157 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0011830 con 0x7f51d415a7e0 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51d37fe700 1 -- 172.21.15.84:0/924242157 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c0011990 con 0x7f51d415a7e0 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51d37fe700 1 -- 172.21.15.84:0/924242157 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0011d20 con 0x7f51d415a7e0 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 -- 172.21.15.84:0/924242157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d415a7e0 msgr2=0x7f51d415ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 --2- 172.21.15.84:0/924242157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d415a7e0 0x7f51d415ac30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f51d40ce6a0 tx=0x7f51c0009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 -- 172.21.15.84:0/924242157 shutdown_connections 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 --2- 172.21.15.84:0/924242157 >> v2:172.21.15.107:3300/0 conn(0x7f51d4154910 0x7f51d4070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 --2- 172.21.15.84:0/924242157 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d415a7e0 0x7f51d415ac30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 -- 172.21.15.84:0/924242157 >> 172.21.15.84:0/924242157 conn(0x7f51d40cf3d0 msgr2=0x7f51d40cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:13.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 -- 172.21.15.84:0/924242157 shutdown_connections 2023-07-06T21:40:13.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.167+0000 7f51dabff700 1 -- 172.21.15.84:0/924242157 wait complete. 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.169+0000 7f51dabff700 1 Processor -- start 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.169+0000 7f51dabff700 1 -- start start 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51dabff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40c8d00 0x7f51d414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51dabff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 0x7f51d40b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51dabff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d415a7e0 0x7f51d40c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51dabff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40c7490 con 0x7f51d4154910 2023-07-06T21:40:13.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51dabff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40b3030 con 0x7f51d40c8d00 2023-07-06T21:40:13.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51dabff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d40b3310 con 0x7f51d415a7e0 2023-07-06T21:40:13.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51d3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 0x7f51d40b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51d3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 0x7f51d40b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43392/0 (socket says 172.21.15.84:43392) 2023-07-06T21:40:13.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.170+0000 7f51d3fff700 1 -- 172.21.15.84:0/2914311423 learned_addr learned my addr 172.21.15.84:0/2914311423 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d919c700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d415a7e0 0x7f51d40c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d899b700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40c8d00 0x7f51d414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d3fff700 1 -- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d415a7e0 msgr2=0x7f51d40c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d3fff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d415a7e0 0x7f51d40c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d3fff700 1 -- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40c8d00 msgr2=0x7f51d414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d3fff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40c8d00 0x7f51d414f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d3fff700 1 -- 172.21.15.84:0/2914311423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00094a0 con 0x7f51d4154910 2023-07-06T21:40:13.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d3fff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 0x7f51d40b26e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f51c0000c00 tx=0x7f51c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:13.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c00036a0 con 0x7f51d4154910 2023-07-06T21:40:13.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c001e070 con 0x7f51d4154910 2023-07-06T21:40:13.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c0003a60 con 0x7f51d4154910 2023-07-06T21:40:13.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d40c70b0 con 0x7f51d4154910 2023-07-06T21:40:13.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.171+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d414f700 con 0x7f51d4154910 2023-07-06T21:40:13.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.173+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d40c75d0 con 0x7f51d4154910 2023-07-06T21:40:13.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.173+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f51c0003bc0 con 0x7f51d4154910 2023-07-06T21:40:13.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.175+0000 7f51d17fa700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f51b806ace0 0x7f51b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.175+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f51c008a520 con 0x7f51d4154910 2023-07-06T21:40:13.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.176+0000 7f51d899b700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f51b806ace0 0x7f51b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.176+0000 7f51d899b700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f51b806ace0 0x7f51b806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51d4159f90 tx=0x7f51c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:13.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.179+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f51d40c75d0 con 0x7f51d4154910 2023-07-06T21:40:13.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.374+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51d40c75d0 con 0x7f51d4154910 2023-07-06T21:40:13.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.376+0000 7f51d17fa700 1 -- 172.21.15.84:0/2914311423 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f51d40c75d0 con 0x7f51d4154910 2023-07-06T21:40:13.378 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:13.378 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:13.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f51b806ace0 msgr2=0x7f51b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f51b806ace0 0x7f51b806d190 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51d4159f90 tx=0x7f51c8008040 comp rx=0 tx=0).stop 2023-07-06T21:40:13.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 msgr2=0x7f51d40b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 0x7f51d40b26e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f51c0000c00 tx=0x7f51c0004750 comp rx=0 tx=0).stop 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 shutdown_connections 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f51b806ace0 0x7f51b806d190 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51d415a7e0 0x7f51d40c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51d40c8d00 0x7f51d414f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 --2- 172.21.15.84:0/2914311423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f51d4154910 0x7f51d40b26e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 >> 172.21.15.84:0/2914311423 conn(0x7f51d40cf3d0 msgr2=0x7f51d40cb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:13.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 shutdown_connections 2023-07-06T21:40:13.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.378+0000 7f51dabff700 1 -- 172.21.15.84:0/2914311423 wait complete. 2023-07-06T21:40:13.398 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:13.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:13.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.593+0000 7f85d11ff700 1 Processor -- start 2023-07-06T21:40:13.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.593+0000 7f85d11ff700 1 -- start start 2023-07-06T21:40:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85d11ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85d11ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85cc0b1a80 0x7f85cc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85d11ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85cc0d1830 con 0x7f85cc0ceda0 2023-07-06T21:40:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85d11ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85cc0cf330 con 0x7f85cc0b1a80 2023-07-06T21:40:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85d11ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc0d1270 con 0x7f85cc0b81c0 2023-07-06T21:40:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85c3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f85cc0b1a80 0x7f85cc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85cb59e700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f85cc0ceda0 0x7f85cc15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36270/0 (socket says 172.21.15.84:36270) 2023-07-06T21:40:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85cb59e700 1 -- 172.21.15.84:0/1182120057 learned_addr learned my addr 172.21.15.84:0/1182120057 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:13.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.594+0000 7f85cad9d700 1 --2- 172.21.15.84:0/1182120057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85c3fff700 1 -- 172.21.15.84:0/1182120057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 msgr2=0x7f85cc0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:13.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85c3fff700 1 --2- 172.21.15.84:0/1182120057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85c3fff700 1 -- 172.21.15.84:0/1182120057 >> v1:172.21.15.84:6789/0 conn(0x7f85cc0ceda0 legacy=0x7f85cc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85c3fff700 1 -- 172.21.15.84:0/1182120057 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc0d1550 con 0x7f85cc0b1a80 2023-07-06T21:40:13.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85ca59c700 1 -- 172.21.15.84:0/1182120057 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149835854 0 0) 0x7f85cc0d1830 con 0x7f85cc0ceda0 2023-07-06T21:40:13.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85c3fff700 1 --2- 172.21.15.84:0/1182120057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cfb40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009840 tx=0x7f85b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe724f300bcbf0c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:13.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85ca59c700 1 -- 172.21.15.84:0/1182120057 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4011830 con 0x7f85cc0b1a80 2023-07-06T21:40:13.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85ca59c700 1 -- 172.21.15.84:0/1182120057 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4011990 con 0x7f85cc0b1a80 2023-07-06T21:40:13.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.595+0000 7f85ca59c700 1 -- 172.21.15.84:0/1182120057 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4011d20 con 0x7f85cc0b1a80 2023-07-06T21:40:13.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 -- 172.21.15.84:0/1182120057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 msgr2=0x7f85cc0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 --2- 172.21.15.84:0/1182120057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cfb40 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009840 tx=0x7f85b4009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:13.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 -- 172.21.15.84:0/1182120057 shutdown_connections 2023-07-06T21:40:13.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 --2- 172.21.15.84:0/1182120057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 --2- 172.21.15.84:0/1182120057 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cfb40 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 -- 172.21.15.84:0/1182120057 >> 172.21.15.84:0/1182120057 conn(0x7f85cc06b2b0 msgr2=0x7f85cc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:13.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 -- 172.21.15.84:0/1182120057 shutdown_connections 2023-07-06T21:40:13.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.596+0000 7f85d11ff700 1 -- 172.21.15.84:0/1182120057 wait complete. 2023-07-06T21:40:13.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.598+0000 7f85d11ff700 1 Processor -- start 2023-07-06T21:40:13.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.598+0000 7f85d11ff700 1 -- start start 2023-07-06T21:40:13.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85d11ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85d11ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc1507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85d11ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85cc0ceda0 0x7f85cc0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85d11ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc158b10 con 0x7f85cc0b1a80 2023-07-06T21:40:13.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85d11ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc1511b0 con 0x7f85cc0ceda0 2023-07-06T21:40:13.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85d11ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85cc151490 con 0x7f85cc0b81c0 2023-07-06T21:40:13.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43410/0 (socket says 172.21.15.84:43410) 2023-07-06T21:40:13.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 -- 172.21.15.84:0/447999904 learned_addr learned my addr 172.21.15.84:0/447999904 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:13.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 -- 172.21.15.84:0/447999904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 msgr2=0x7f85cc1507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc1507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 -- 172.21.15.84:0/447999904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85cc0ceda0 msgr2=0x7f85cc0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85cc0ceda0 0x7f85cc0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.599+0000 7f85cad9d700 1 -- 172.21.15.84:0/447999904 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b40094a0 con 0x7f85cc0b1a80 2023-07-06T21:40:13.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.600+0000 7f85cad9d700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cb9a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e9b0 tx=0x7f85bc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.600+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00c850 con 0x7f85cc0b1a80 2023-07-06T21:40:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.600+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc00c9b0 con 0x7f85cc0b1a80 2023-07-06T21:40:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.600+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85cc151770 con 0x7f85cc0b1a80 2023-07-06T21:40:13.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.600+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc00cd20 con 0x7f85cc0b1a80 2023-07-06T21:40:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.600+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85cc0cbee0 con 0x7f85cc0b1a80 2023-07-06T21:40:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.601+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85cc06bbb0 con 0x7f85cc0b1a80 2023-07-06T21:40:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.602+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85bc013470 con 0x7f85cc0b1a80 2023-07-06T21:40:13.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.603+0000 7f85c37fe700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85ac06ace0 0x7f85ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.603+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f85bc089610 con 0x7f85cc0b1a80 2023-07-06T21:40:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.603+0000 7f85c3fff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85ac06ace0 0x7f85ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.603+0000 7f85c3fff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85ac06ace0 0x7f85ac06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85cc1e7010 tx=0x7f85b4010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:13.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.607+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85bc054670 con 0x7f85cc0b1a80 2023-07-06T21:40:13.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.805+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85cc04e1d0 con 0x7f85cc0b1a80 2023-07-06T21:40:13.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.805+0000 7f85c37fe700 1 -- 172.21.15.84:0/447999904 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85bc057c90 con 0x7f85cc0b1a80 2023-07-06T21:40:13.808 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:13.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.807+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85ac06ace0 msgr2=0x7f85ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.807+0000 7f85d11ff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85ac06ace0 0x7f85ac06d190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85cc1e7010 tx=0x7f85b4010000 comp rx=0 tx=0).stop 2023-07-06T21:40:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.807+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 msgr2=0x7f85cc0cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.807+0000 7f85d11ff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cb9a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00e9b0 tx=0x7f85bc00ed70 comp rx=0 tx=0).stop 2023-07-06T21:40:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 shutdown_connections 2023-07-06T21:40:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85ac06ace0 0x7f85ac06d190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85cc0ceda0 0x7f85cc0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85cc0b1a80 0x7f85cc0cb9a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 --2- 172.21.15.84:0/447999904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85cc0b81c0 0x7f85cc1507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:13.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 >> 172.21.15.84:0/447999904 conn(0x7f85cc06b2b0 msgr2=0x7f85cc0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:13.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 shutdown_connections 2023-07-06T21:40:13.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:13.808+0000 7f85d11ff700 1 -- 172.21.15.84:0/447999904 wait complete. 2023-07-06T21:40:13.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:14.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.025+0000 7f6dd87ff700 1 Processor -- start 2023-07-06T21:40:14.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.026+0000 7f6dd87ff700 1 -- start start 2023-07-06T21:40:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd87ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6dd00ca8c0 0x7f6dd00703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd87ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dd00708e0 0x7f6dd006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd87ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd0070ce0 con 0x7f6dd0159220 2023-07-06T21:40:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd87ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00cad60 con 0x7f6dd00708e0 2023-07-06T21:40:14.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd87ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00cf600 con 0x7f6dd00ca8c0 2023-07-06T21:40:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd6d9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6dd00708e0 0x7f6dd006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd659b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6dd0159220 0x7f6dd00cc3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36286/0 (socket says 172.21.15.84:36286) 2023-07-06T21:40:14.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd659b700 1 -- 172.21.15.84:0/3803283333 learned_addr learned my addr 172.21.15.84:0/3803283333 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/3803283333 >> v2:172.21.15.107:3300/0 conn(0x7f6dd00ca8c0 msgr2=0x7f6dd00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd5d9a700 1 --2- 172.21.15.84:0/3803283333 >> v2:172.21.15.107:3300/0 conn(0x7f6dd00ca8c0 0x7f6dd00703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd5599700 1 -- 172.21.15.84:0/3803283333 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3691064479 0 0) 0x7f6dd0070ce0 con 0x7f6dd0159220 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd6d9c700 1 --2- 172.21.15.84:0/3803283333 >> v2:172.21.15.107:3300/0 conn(0x7f6dd00ca8c0 0x7f6dd00703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/3803283333 >> v1:172.21.15.84:6789/0 conn(0x7f6dd0159220 legacy=0x7f6dd00cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/3803283333 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00ccf40 con 0x7f6dd00708e0 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.027+0000 7f6dd5d9a700 1 --2- 172.21.15.84:0/3803283333 >> v2:172.21.15.107:3300/0 conn(0x7f6dd00ca8c0 0x7f6dd00703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd6d9c700 1 --2- 172.21.15.84:0/3803283333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd00708e0 0x7f6dd006c400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ad690 tx=0x7f6dc800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75651937ab22b2da server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd5599700 1 -- 172.21.15.84:0/3803283333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc800e070 con 0x7f6dd00708e0 2023-07-06T21:40:14.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd5599700 1 -- 172.21.15.84:0/3803283333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc800bbd0 con 0x7f6dd00708e0 2023-07-06T21:40:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd5599700 1 -- 172.21.15.84:0/3803283333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8012430 con 0x7f6dd00708e0 2023-07-06T21:40:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd87ff700 1 -- 172.21.15.84:0/3803283333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd00708e0 msgr2=0x7f6dd006c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/3803283333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd00708e0 0x7f6dd006c400 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6dd00ad690 tx=0x7f6dc800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd87ff700 1 -- 172.21.15.84:0/3803283333 shutdown_connections 2023-07-06T21:40:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/3803283333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd00708e0 0x7f6dd006c400 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/3803283333 >> v2:172.21.15.107:3300/0 conn(0x7f6dd00ca8c0 0x7f6dd00703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.028+0000 7f6dd87ff700 1 -- 172.21.15.84:0/3803283333 >> 172.21.15.84:0/3803283333 conn(0x7f6dd00cbf10 msgr2=0x7f6dd00ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.029+0000 7f6dd87ff700 1 -- 172.21.15.84:0/3803283333 shutdown_connections 2023-07-06T21:40:14.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.029+0000 7f6dd87ff700 1 -- 172.21.15.84:0/3803283333 wait complete. 2023-07-06T21:40:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.030+0000 7f6dd87ff700 1 Processor -- start 2023-07-06T21:40:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.030+0000 7f6dd87ff700 1 -- start start 2023-07-06T21:40:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.031+0000 7f6dd87ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6dd00708e0 0x7f6dd0151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.031+0000 7f6dd87ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dd00ca8c0 0x7f6dd01f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.031+0000 7f6dd87ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 0x7f6dd00afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.031+0000 7f6dd87ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01523a0 con 0x7f6dd0159220 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.031+0000 7f6dd87ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01f00b0 con 0x7f6dd00ca8c0 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.031+0000 7f6dd87ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01f0390 con 0x7f6dd00708e0 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 0x7f6dd00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 0x7f6dd00afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43440/0 (socket says 172.21.15.84:43440) 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/616517428 learned_addr learned my addr 172.21.15.84:0/616517428 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd659b700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6dd00708e0 0x7f6dd0151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd5d9a700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dd00ca8c0 0x7f6dd01f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6dd00708e0 msgr2=0x7f6dd0151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6dd00708e0 0x7f6dd0151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dd00ca8c0 msgr2=0x7f6dd01f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dd00ca8c0 0x7f6dd01f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 -- 172.21.15.84:0/616517428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8009a70 con 0x7f6dd0159220 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd5d9a700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dd00ca8c0 0x7f6dd01f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:14.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.032+0000 7f6dd6d9c700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 0x7f6dd00afd00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6dd006c1c0 tx=0x7f6dc8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.033+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8012430 con 0x7f6dd0159220 2023-07-06T21:40:14.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.033+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc8020070 con 0x7f6dd0159220 2023-07-06T21:40:14.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.033+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd01f0670 con 0x7f6dd0159220 2023-07-06T21:40:14.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.033+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8004280 con 0x7f6dd0159220 2023-07-06T21:40:14.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.033+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd01efcd0 con 0x7f6dd0159220 2023-07-06T21:40:14.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.034+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd004e1d0 con 0x7f6dd0159220 2023-07-06T21:40:14.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.035+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6dc80043e0 con 0x7f6dd0159220 2023-07-06T21:40:14.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.036+0000 7f6dc6ffd700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6db406ad10 0x7f6db406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.036+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6dc8022070 con 0x7f6dd0159220 2023-07-06T21:40:14.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.036+0000 7f6dd659b700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6db406ad10 0x7f6db406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.037+0000 7f6dd659b700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6db406ad10 0x7f6db406d1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007fd0 tx=0x7f6dcc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.040+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6dc8056750 con 0x7f6dd0159220 2023-07-06T21:40:14.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.239+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6dd00cf010 con 0x7f6dd0159220 2023-07-06T21:40:14.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.240+0000 7f6dc6ffd700 1 -- 172.21.15.84:0/616517428 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6dc801ae60 con 0x7f6dd0159220 2023-07-06T21:40:14.242 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:14.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6db406ad10 msgr2=0x7f6db406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6db406ad10 0x7f6db406d1c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc007fd0 tx=0x7f6dcc007de0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 msgr2=0x7f6dd00afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 0x7f6dd00afd00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6dd006c1c0 tx=0x7f6dc8004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 shutdown_connections 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6db406ad10 0x7f6db406d1c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6dd0159220 0x7f6dd00afd00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6dd00708e0 0x7f6dd0151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 --2- 172.21.15.84:0/616517428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6dd00ca8c0 0x7f6dd01f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.242+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 >> 172.21.15.84:0/616517428 conn(0x7f6dd00cbf10 msgr2=0x7f6dd00734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.243+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 shutdown_connections 2023-07-06T21:40:14.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.243+0000 7f6dd87ff700 1 -- 172.21.15.84:0/616517428 wait complete. 2023-07-06T21:40:14.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:14.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.457+0000 7f585119d700 1 Processor -- start 2023-07-06T21:40:14.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.457+0000 7f585119d700 1 -- start start 2023-07-06T21:40:14.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.458+0000 7f585119d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0b2e40 0x7f584c0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.458+0000 7f585119d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f584c0cfb70 0x7f584c0d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.458+0000 7f585119d700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584c0b3e50 con 0x7f584c0d24e0 2023-07-06T21:40:14.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.458+0000 7f585119d700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584c15ca50 con 0x7f584c0cfb70 2023-07-06T21:40:14.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.458+0000 7f585119d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0b3b70 con 0x7f584c0b2e40 2023-07-06T21:40:14.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.458+0000 7f584b7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f584c0cfb70 0x7f584c0d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f584c0cfb70 0x7f584c0d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43456/0 (socket says 172.21.15.84:43456) 2023-07-06T21:40:14.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 -- 172.21.15.84:0/3516998555 learned_addr learned my addr 172.21.15.84:0/3516998555 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:14.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584bfff700 1 --2- 172.21.15.84:0/3516998555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0b2e40 0x7f584c0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 -- 172.21.15.84:0/3516998555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0b2e40 msgr2=0x7f584c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:14.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 --2- 172.21.15.84:0/3516998555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0b2e40 0x7f584c0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 -- 172.21.15.84:0/3516998555 >> v1:172.21.15.107:6789/0 conn(0x7f584c0d24e0 legacy=0x7f584c0b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 -- 172.21.15.84:0/3516998555 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584c0ceb50 con 0x7f584c0cfb70 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584b7fe700 1 --2- 172.21.15.84:0/3516998555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c0d0580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58400098f0 tx=0x7f5840009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72315d35c4f950fd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.459+0000 7f584affd700 1 -- 172.21.15.84:0/3516998555 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840011830 con 0x7f584c0cfb70 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f584affd700 1 -- 172.21.15.84:0/3516998555 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5840011990 con 0x7f584c0cfb70 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f584affd700 1 -- 172.21.15.84:0/3516998555 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840011d20 con 0x7f584c0cfb70 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 -- 172.21.15.84:0/3516998555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 msgr2=0x7f584c0d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 --2- 172.21.15.84:0/3516998555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c0d0580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f58400098f0 tx=0x7f5840009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 -- 172.21.15.84:0/3516998555 shutdown_connections 2023-07-06T21:40:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 --2- 172.21.15.84:0/3516998555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0b2e40 0x7f584c0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 --2- 172.21.15.84:0/3516998555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c0d0580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 -- 172.21.15.84:0/3516998555 >> 172.21.15.84:0/3516998555 conn(0x7f584c09bcf0 msgr2=0x7f584c0d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 -- 172.21.15.84:0/3516998555 shutdown_connections 2023-07-06T21:40:14.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.460+0000 7f585119d700 1 -- 172.21.15.84:0/3516998555 wait complete. 2023-07-06T21:40:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.462+0000 7f585119d700 1 Processor -- start 2023-07-06T21:40:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.462+0000 7f585119d700 1 -- start start 2023-07-06T21:40:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f585119d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f584c0b2e40 0x7f584c0c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f585119d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f585119d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0d24e0 0x7f584c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f585119d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0c8770 con 0x7f584c0cfb70 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f585119d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c0c7df0 con 0x7f584c0b2e40 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f585119d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584c1577f0 con 0x7f584c0d24e0 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43458/0 (socket says 172.21.15.84:43458) 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f584b7fe700 1 -- 172.21.15.84:0/2717423049 learned_addr learned my addr 172.21.15.84:0/2717423049 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.463+0000 7f584bfff700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f584c0b2e40 0x7f584c0c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f584b7fe700 1 -- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0d24e0 msgr2=0x7f584c1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f584b7fe700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0d24e0 0x7f584c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f584b7fe700 1 -- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f584c0b2e40 msgr2=0x7f584c0c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f584b7fe700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f584c0b2e40 0x7f584c0c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f584b7fe700 1 -- 172.21.15.84:0/2717423049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840009550 con 0x7f584c0cfb70 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f584b7fe700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c156de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5840009c40 tx=0x7f5840004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400036a0 con 0x7f584c0cfb70 2023-07-06T21:40:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584001e070 con 0x7f584c0cfb70 2023-07-06T21:40:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840003a60 con 0x7f584c0cfb70 2023-07-06T21:40:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c0c80b0 con 0x7f584c0cfb70 2023-07-06T21:40:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.464+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c0c90d0 con 0x7f584c0cfb70 2023-07-06T21:40:14.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.466+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584c0d15f0 con 0x7f584c0cfb70 2023-07-06T21:40:14.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.467+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5840003bc0 con 0x7f584c0cfb70 2023-07-06T21:40:14.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.468+0000 7f5848ff9700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f583806ace0 0x7f583806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.468+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f584008a4b0 con 0x7f584c0cfb70 2023-07-06T21:40:14.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.468+0000 7f584bfff700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f583806ace0 0x7f583806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.469+0000 7f584bfff700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f583806ace0 0x7f583806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f583c009ab0 tx=0x7f583c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.472+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5840055510 con 0x7f584c0cfb70 2023-07-06T21:40:14.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.671+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f584c0c92c0 con 0x7f584c0cfb70 2023-07-06T21:40:14.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.672+0000 7f5848ff9700 1 -- 172.21.15.84:0/2717423049 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5840058b30 con 0x7f584c0cfb70 2023-07-06T21:40:14.679 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:14.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f583806ace0 msgr2=0x7f583806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f583806ace0 0x7f583806d190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f583c009ab0 tx=0x7f583c008040 comp rx=0 tx=0).stop 2023-07-06T21:40:14.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 msgr2=0x7f584c156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c156de0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5840009c40 tx=0x7f5840004750 comp rx=0 tx=0).stop 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 shutdown_connections 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f583806ace0 0x7f583806d190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f584c0d24e0 0x7f584c1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f584c0b2e40 0x7f584c0c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 --2- 172.21.15.84:0/2717423049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f584c0cfb70 0x7f584c156de0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 >> 172.21.15.84:0/2717423049 conn(0x7f584c09bcf0 msgr2=0x7f584c0d2990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:14.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 shutdown_connections 2023-07-06T21:40:14.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.674+0000 7f585119d700 1 -- 172.21.15.84:0/2717423049 wait complete. 2023-07-06T21:40:14.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:14.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.890+0000 7f86e83ff700 1 Processor -- start 2023-07-06T21:40:14.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.890+0000 7f86e83ff700 1 -- start start 2023-07-06T21:40:14.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.891+0000 7f86e83ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e83ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e00cf600 con 0x7f86e00cc3d0 2023-07-06T21:40:14.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e83ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e00ccf40 con 0x7f86e0070ab0 2023-07-06T21:40:14.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e83ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00c9b00 con 0x7f86e0159220 2023-07-06T21:40:14.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e599a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f86e00cc3d0 0x7f86e00703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36302/0 (socket says 172.21.15.84:36302) 2023-07-06T21:40:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e599a700 1 -- 172.21.15.84:0/1692572118 learned_addr learned my addr 172.21.15.84:0/1692572118 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e619b700 1 --2- 172.21.15.84:0/1692572118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616069131 0 0) 0x7f86e00cf600 con 0x7f86e00cc3d0 2023-07-06T21:40:14.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86cc0035b0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163003345 0 0) 0x7f86e00ccf40 con 0x7f86e0070ab0 2023-07-06T21:40:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.892+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e00cf600 con 0x7f86e0070ab0 2023-07-06T21:40:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e619b700 1 -- 172.21.15.84:0/1692572118 >> v1:172.21.15.107:6789/0 conn(0x7f86e0070ab0 legacy=0x7f86e006c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e619b700 1 -- 172.21.15.84:0/1692572118 >> v1:172.21.15.84:6789/0 conn(0x7f86e00cc3d0 legacy=0x7f86e00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e619b700 1 -- 172.21.15.84:0/1692572118 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e00c9de0 con 0x7f86e0159220 2023-07-06T21:40:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e619b700 1 --2- 172.21.15.84:0/1692572118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00ca8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009870 tx=0x7f86dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0f7bf81cea5fd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc011830 con 0x7f86e0159220 2023-07-06T21:40:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc011990 con 0x7f86e0159220 2023-07-06T21:40:14.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.893+0000 7f86e5199700 1 -- 172.21.15.84:0/1692572118 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc011d00 con 0x7f86e0159220 2023-07-06T21:40:14.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 -- 172.21.15.84:0/1692572118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 msgr2=0x7f86e00ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 --2- 172.21.15.84:0/1692572118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00ca8c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009870 tx=0x7f86dc009b80 comp rx=0 tx=0).stop 2023-07-06T21:40:14.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 -- 172.21.15.84:0/1692572118 shutdown_connections 2023-07-06T21:40:14.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 --2- 172.21.15.84:0/1692572118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00ca8c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 -- 172.21.15.84:0/1692572118 >> 172.21.15.84:0/1692572118 conn(0x7f86e00cbf10 msgr2=0x7f86e00ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:14.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 -- 172.21.15.84:0/1692572118 shutdown_connections 2023-07-06T21:40:14.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.894+0000 7f86e83ff700 1 -- 172.21.15.84:0/1692572118 wait complete. 2023-07-06T21:40:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.895+0000 7f86e83ff700 1 Processor -- start 2023-07-06T21:40:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.896+0000 7f86e83ff700 1 -- start start 2023-07-06T21:40:14.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.896+0000 7f86e83ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86e0070ab0 0x7f86e0157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.896+0000 7f86e83ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 0x7f86e01f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.896+0000 7f86e83ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e83ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0158520 con 0x7f86e00cc3d0 2023-07-06T21:40:14.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e83ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01e8850 con 0x7f86e0070ab0 2023-07-06T21:40:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e83ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00ae1c0 con 0x7f86e0159220 2023-07-06T21:40:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 0x7f86e01f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 0x7f86e01f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43474/0 (socket says 172.21.15.84:43474) 2023-07-06T21:40:14.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 -- 172.21.15.84:0/3928654666 learned_addr learned my addr 172.21.15.84:0/3928654666 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e619b700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86e0070ab0 0x7f86e0157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e699c700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 -- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 msgr2=0x7f86e00c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 -- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86e0070ab0 msgr2=0x7f86e0157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86e0070ab0 0x7f86e0157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 -- 172.21.15.84:0/3928654666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc0094a0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e699c700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.897+0000 7f86e599a700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 0x7f86e01f3120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c700 tx=0x7f86d000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.898+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0005d70 con 0x7f86e00cc3d0 2023-07-06T21:40:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.898+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d00070d0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.898+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e00ae4a0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.898+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0007460 con 0x7f86e00cc3d0 2023-07-06T21:40:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.898+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e01e8b30 con 0x7f86e00cc3d0 2023-07-06T21:40:14.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.899+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e00ae9c0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.900+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f86d00075c0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.901+0000 7f86d6ffd700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86c406ace0 0x7f86c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.901+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f86d00889c0 con 0x7f86e00cc3d0 2023-07-06T21:40:14.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.901+0000 7f86e619b700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86c406ace0 0x7f86c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.902+0000 7f86e619b700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86c406ace0 0x7f86c406d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0090b0 tx=0x7f86dc01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:14.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:14.905+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f86e00ae9c0 con 0x7f86e00cc3d0 2023-07-06T21:40:15.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.102+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86e00ae9c0 con 0x7f86e00cc3d0 2023-07-06T21:40:15.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.103+0000 7f86d6ffd700 1 -- 172.21.15.84:0/3928654666 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86e00ae9c0 con 0x7f86e00cc3d0 2023-07-06T21:40:15.105 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86c406ace0 msgr2=0x7f86c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86c406ace0 0x7f86c406d190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86dc0090b0 tx=0x7f86dc01c040 comp rx=0 tx=0).stop 2023-07-06T21:40:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 msgr2=0x7f86e01f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 0x7f86e01f3120 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f86d000c700 tx=0x7f86d000ca10 comp rx=0 tx=0).stop 2023-07-06T21:40:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 shutdown_connections 2023-07-06T21:40:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f86c406ace0 0x7f86c406d190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f86e0159220 0x7f86e00c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f86e0070ab0 0x7f86e0157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 --2- 172.21.15.84:0/3928654666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86e00cc3d0 0x7f86e01f3120 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 >> 172.21.15.84:0/3928654666 conn(0x7f86e00cbf10 msgr2=0x7f86e0069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 shutdown_connections 2023-07-06T21:40:15.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.105+0000 7f86e83ff700 1 -- 172.21.15.84:0/3928654666 wait complete. 2023-07-06T21:40:15.125 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:15.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.318+0000 7f92fd3ff700 1 Processor -- start 2023-07-06T21:40:15.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.318+0000 7f92fd3ff700 1 -- start start 2023-07-06T21:40:15.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92fd3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92f80d1580 0x7f92f8166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92fd3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f80d0990 con 0x7f92f80d3580 2023-07-06T21:40:15.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92fd3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f80cd9d0 con 0x7f92f80b8530 2023-07-06T21:40:15.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92fd3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f80d3af0 con 0x7f92f80d1580 2023-07-06T21:40:15.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92f77fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92f80d1580 0x7f92f8166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92f67fc700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f92f80d3580 0x7f92f80d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36308/0 (socket says 172.21.15.84:36308) 2023-07-06T21:40:15.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.319+0000 7f92f67fc700 1 -- 172.21.15.84:0/1239696118 learned_addr learned my addr 172.21.15.84:0/1239696118 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:15.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f77fe700 1 -- 172.21.15.84:0/1239696118 >> v1:172.21.15.107:6789/0 conn(0x7f92f80b8530 legacy=0x7f92f80cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f77fe700 1 -- 172.21.15.84:0/1239696118 >> v1:172.21.15.84:6789/0 conn(0x7f92f80d3580 legacy=0x7f92f80d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f77fe700 1 -- 172.21.15.84:0/1239696118 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80cd410 con 0x7f92f80d1580 2023-07-06T21:40:15.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f5ffb700 1 -- 172.21.15.84:0/1239696118 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213534681 0 0) 0x7f92f80d0990 con 0x7f92f80d3580 2023-07-06T21:40:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f77fe700 1 --2- 172.21.15.84:0/1239696118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d1580 0x7f92f8166ff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009d90 tx=0x7f92f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d5077980690131 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f5ffb700 1 -- 172.21.15.84:0/1239696118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f000e070 con 0x7f92f80d1580 2023-07-06T21:40:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f5ffb700 1 -- 172.21.15.84:0/1239696118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f000bbd0 con 0x7f92f80d1580 2023-07-06T21:40:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.320+0000 7f92f5ffb700 1 -- 172.21.15.84:0/1239696118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f0012430 con 0x7f92f80d1580 2023-07-06T21:40:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 -- 172.21.15.84:0/1239696118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d1580 msgr2=0x7f92f8166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/1239696118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d1580 0x7f92f8166ff0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f92f0009d90 tx=0x7f92f000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 -- 172.21.15.84:0/1239696118 shutdown_connections 2023-07-06T21:40:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/1239696118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d1580 0x7f92f8166ff0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 -- 172.21.15.84:0/1239696118 >> 172.21.15.84:0/1239696118 conn(0x7f92f80715c0 msgr2=0x7f92f8071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 -- 172.21.15.84:0/1239696118 shutdown_connections 2023-07-06T21:40:15.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.321+0000 7f92fd3ff700 1 -- 172.21.15.84:0/1239696118 wait complete. 2023-07-06T21:40:15.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.323+0000 7f92fd3ff700 1 Processor -- start 2023-07-06T21:40:15.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.323+0000 7f92fd3ff700 1 -- start start 2023-07-06T21:40:15.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92fd3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f80b8530 0x7f92f8150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92fd3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f80d1580 0x7f92f815c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92fd3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 0x7f92f80a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92fd3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81f21a0 con 0x7f92f80d3580 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92fd3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81505d0 con 0x7f92f80d1580 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92fd3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f81f23f0 con 0x7f92f80b8530 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 0x7f92f80a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92f77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 0x7f92f80a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43504/0 (socket says 172.21.15.84:43504) 2023-07-06T21:40:15.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92f77fe700 1 -- 172.21.15.84:0/100617533 learned_addr learned my addr 172.21.15.84:0/100617533 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:15.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92f67fc700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f80d1580 0x7f92f815c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.324+0000 7f92f6ffd700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f80b8530 0x7f92f8150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f77fe700 1 -- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f80b8530 msgr2=0x7f92f8150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f77fe700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f80b8530 0x7f92f8150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f77fe700 1 -- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f80d1580 msgr2=0x7f92f815c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f77fe700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f80d1580 0x7f92f815c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f77fe700 1 -- 172.21.15.84:0/100617533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f0009a70 con 0x7f92f80d3580 2023-07-06T21:40:15.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f6ffd700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f80b8530 0x7f92f8150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:15.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92f77fe700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 0x7f92f80a9c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92f80d3cc0 tx=0x7f92f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:15.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f001b820 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f0020070 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f81f26d0 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f001bbc0 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.325+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f81f2c90 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.326+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f80008d0 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.327+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92f001bd20 con 0x7f92f80d3580 2023-07-06T21:40:15.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.328+0000 7f92df7fe700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92e406ace0 0x7f92e406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.328+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f92f008b370 con 0x7f92f80d3580 2023-07-06T21:40:15.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.328+0000 7f92f6ffd700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92e406ace0 0x7f92e406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.329+0000 7f92f6ffd700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92e406ace0 0x7f92e406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e8000fb0 tx=0x7f92e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:15.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.332+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92f0059e60 con 0x7f92f80d3580 2023-07-06T21:40:15.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.543+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92f806a250 con 0x7f92f80d3580 2023-07-06T21:40:15.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.544+0000 7f92df7fe700 1 -- 172.21.15.84:0/100617533 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f92f00599f0 con 0x7f92f80d3580 2023-07-06T21:40:15.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.546+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92e406ace0 msgr2=0x7f92e406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.546+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92e406ace0 0x7f92e406d190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f92e8000fb0 tx=0x7f92e8009040 comp rx=0 tx=0).stop 2023-07-06T21:40:15.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.546+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 msgr2=0x7f92f80a9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.546+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 0x7f92f80a9c00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f92f80d3cc0 tx=0x7f92f0004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 shutdown_connections 2023-07-06T21:40:15.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92e406ace0 0x7f92e406d190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92f80d3580 0x7f92f80a9c00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f80b8530 0x7f92f8150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 --2- 172.21.15.84:0/100617533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f80d1580 0x7f92f815c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 >> 172.21.15.84:0/100617533 conn(0x7f92f80715c0 msgr2=0x7f92f815eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:15.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 shutdown_connections 2023-07-06T21:40:15.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.547+0000 7f92fd3ff700 1 -- 172.21.15.84:0/100617533 wait complete. 2023-07-06T21:40:15.551 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:15.567 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:40:15.567 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:15.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:15.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.840+0000 7f58f8fff700 1 Processor -- start 2023-07-06T21:40:15.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.840+0000 7f58f8fff700 1 -- start start 2023-07-06T21:40:15.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.841+0000 7f58f8fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58f40d3710 0x7f58f40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.841+0000 7f58f8fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f58f40cfa90 0x7f58f40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.841+0000 7f58f8fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40cd910 con 0x7f58f40b83c0 2023-07-06T21:40:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.841+0000 7f58f8fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40cff10 con 0x7f58f40d3710 2023-07-06T21:40:15.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.841+0000 7f58f8fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40d0a80 con 0x7f58f40cfa90 2023-07-06T21:40:15.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58f40d3710 0x7f58f40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f58f40d3710 0x7f58f40cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43514/0 (socket says 172.21.15.84:43514) 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 -- 172.21.15.84:0/2531640591 learned_addr learned my addr 172.21.15.84:0/2531640591 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58f2d9d700 1 --2- 172.21.15.84:0/2531640591 >> v2:172.21.15.107:3300/0 conn(0x7f58f40cfa90 0x7f58f40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 -- 172.21.15.84:0/2531640591 >> v2:172.21.15.107:3300/0 conn(0x7f58f40cfa90 msgr2=0x7f58f40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 --2- 172.21.15.84:0/2531640591 >> v2:172.21.15.107:3300/0 conn(0x7f58f40cfa90 0x7f58f40d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 -- 172.21.15.84:0/2531640591 >> v1:172.21.15.107:6789/0 conn(0x7f58f40b83c0 legacy=0x7f58f40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.842+0000 7f58ebfff700 1 -- 172.21.15.84:0/2531640591 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40b8790 con 0x7f58f40d3710 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58ebfff700 1 --2- 172.21.15.84:0/2531640591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f40cf550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58f40b1b00 tx=0x7f58dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708db4c98c7b1a71 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f1d9b700 1 -- 172.21.15.84:0/2531640591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc011830 con 0x7f58f40d3710 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f1d9b700 1 -- 172.21.15.84:0/2531640591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc011990 con 0x7f58f40d3710 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f1d9b700 1 -- 172.21.15.84:0/2531640591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc011d20 con 0x7f58f40d3710 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f8fff700 1 -- 172.21.15.84:0/2531640591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 msgr2=0x7f58f40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f8fff700 1 --2- 172.21.15.84:0/2531640591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f40cf550 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f58f40b1b00 tx=0x7f58dc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f8fff700 1 -- 172.21.15.84:0/2531640591 shutdown_connections 2023-07-06T21:40:15.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f8fff700 1 --2- 172.21.15.84:0/2531640591 >> v2:172.21.15.107:3300/0 conn(0x7f58f40cfa90 0x7f58f40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f8fff700 1 --2- 172.21.15.84:0/2531640591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f40cf550 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.843+0000 7f58f8fff700 1 -- 172.21.15.84:0/2531640591 >> 172.21.15.84:0/2531640591 conn(0x7f58f406b350 msgr2=0x7f58f406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:15.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.844+0000 7f58f8fff700 1 -- 172.21.15.84:0/2531640591 shutdown_connections 2023-07-06T21:40:15.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.844+0000 7f58f8fff700 1 -- 172.21.15.84:0/2531640591 wait complete. 2023-07-06T21:40:15.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.846+0000 7f58f8fff700 1 Processor -- start 2023-07-06T21:40:15.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.846+0000 7f58f8fff700 1 -- start start 2023-07-06T21:40:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.847+0000 7f58f8fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58f40b83c0 0x7f58f40c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.847+0000 7f58f8fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40cfa90 0x7f58f40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.847+0000 7f58f8fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f415f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.847+0000 7f58f8fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40c5e50 con 0x7f58f40d3710 2023-07-06T21:40:15.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.847+0000 7f58f8fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40cb640 con 0x7f58f40b83c0 2023-07-06T21:40:15.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.847+0000 7f58f8fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f415dfd0 con 0x7f58f40cfa90 2023-07-06T21:40:15.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f415f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f415f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43526/0 (socket says 172.21.15.84:43526) 2023-07-06T21:40:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 -- 172.21.15.84:0/333594580 learned_addr learned my addr 172.21.15.84:0/333594580 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f259c700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58f40b83c0 0x7f58f40c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58ebfff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40cfa90 0x7f58f40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 -- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40cfa90 msgr2=0x7f58f40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40cfa90 0x7f58f40cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 -- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58f40b83c0 msgr2=0x7f58f40c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:15.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58f40b83c0 0x7f58f40c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 -- 172.21.15.84:0/333594580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc009550 con 0x7f58f40d3710 2023-07-06T21:40:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f2d9d700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f415f2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009d30 tx=0x7f58ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec00cc90 con 0x7f58f40d3710 2023-07-06T21:40:15.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec00f040 con 0x7f58f40d3710 2023-07-06T21:40:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.848+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f415ee10 con 0x7f58f40d3710 2023-07-06T21:40:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.849+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec009e80 con 0x7f58f40d3710 2023-07-06T21:40:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.849+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f415e200 con 0x7f58f40d3710 2023-07-06T21:40:15.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.850+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f415efa0 con 0x7f58f40d3710 2023-07-06T21:40:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.851+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f58ec014480 con 0x7f58f40d3710 2023-07-06T21:40:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.852+0000 7f58eaffd700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f58d406ace0 0x7f58d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.852+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f58ec089c10 con 0x7f58f40d3710 2023-07-06T21:40:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.852+0000 7f58f259c700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f58d406ace0 0x7f58d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:15.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.853+0000 7f58f259c700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f58d406ace0 0x7f58d406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58e4007fd0 tx=0x7f58e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:15.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:15.856+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f58ec054c70 con 0x7f58f40d3710 2023-07-06T21:40:16.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.057+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58f40b42e0 con 0x7f58f40d3710 2023-07-06T21:40:16.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.058+0000 7f58eaffd700 1 -- 172.21.15.84:0/333594580 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f58ec018090 con 0x7f58f40d3710 2023-07-06T21:40:16.061 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:16.061 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:16.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f58d406ace0 msgr2=0x7f58d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f58d406ace0 0x7f58d406d190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f58e4007fd0 tx=0x7f58e4009040 comp rx=0 tx=0).stop 2023-07-06T21:40:16.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 msgr2=0x7f58f415f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f415f2a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009d30 tx=0x7f58ec00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 shutdown_connections 2023-07-06T21:40:16.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f58d406ace0 0x7f58d406d190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f58f40d3710 0x7f58f415f2a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58f40b83c0 0x7f58f40c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 --2- 172.21.15.84:0/333594580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40cfa90 0x7f58f40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 >> 172.21.15.84:0/333594580 conn(0x7f58f406b350 msgr2=0x7f58f40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:16.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 shutdown_connections 2023-07-06T21:40:16.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.060+0000 7f58f8fff700 1 -- 172.21.15.84:0/333594580 wait complete. 2023-07-06T21:40:16.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:16.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.276+0000 7f3d95dff700 1 Processor -- start 2023-07-06T21:40:16.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.277+0000 7f3d95dff700 1 -- start start 2023-07-06T21:40:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d95dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 0x7f3d900d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d95dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900d1830 con 0x7f3d900b1a80 2023-07-06T21:40:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d95dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900cf330 con 0x7f3d900b81c0 2023-07-06T21:40:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d95dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900d1270 con 0x7f3d900cfd10 2023-07-06T21:40:16.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d8ffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3d900b1a80 0x7f3d900d28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36318/0 (socket says 172.21.15.84:36318) 2023-07-06T21:40:16.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d8ffff700 1 -- 172.21.15.84:0/1866347961 learned_addr learned my addr 172.21.15.84:0/1866347961 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d8effd700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 0x7f3d900d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.278+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913120147 0 0) 0x7f3d900d1830 con 0x7f3d900b1a80 2023-07-06T21:40:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d780035b0 con 0x7f3d900b1a80 2023-07-06T21:40:16.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873812539 0 0) 0x7f3d900cf330 con 0x7f3d900b81c0 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d900d1830 con 0x7f3d900b81c0 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892334916 0 0) 0x7f3d780035b0 con 0x7f3d900b1a80 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 msgr2=0x7f3d900d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 0x7f3d900d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 >> v1:172.21.15.107:6789/0 conn(0x7f3d900b81c0 legacy=0x7f3d900cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900d1550 con 0x7f3d900b1a80 2023-07-06T21:40:16.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f3d880019f0 con 0x7f3d900b1a80 2023-07-06T21:40:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 >> v1:172.21.15.84:6789/0 conn(0x7f3d900b1a80 legacy=0x7f3d900d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d78003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d78004850 0x7f3d78004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 0x7f3d78005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900d1270 con 0x7f3d78003bc0 2023-07-06T21:40:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7800bac0 con 0x7f3d78004850 2023-07-06T21:40:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7800bc30 con 0x7f3d900cfd10 2023-07-06T21:40:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d78003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8effd700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d78004850 0x7f3d78004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.279+0000 7f3d8ffff700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 0x7f3d78005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 msgr2=0x7f3d78005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900cfd10 0x7f3d78005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d78004850 msgr2=0x7f3d78004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d78004850 0x7f3d78004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1866347961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d780035b0 con 0x7f3d78003bc0 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8effd700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d78004850 0x7f3d78004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d78003fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001940 tx=0x7f3d78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cc7c5fe8c27eaa server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:16.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80001430 con 0x7f3d78003bc0 2023-07-06T21:40:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d80001d70 con 0x7f3d78003bc0 2023-07-06T21:40:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.280+0000 7f3d8e7fc700 1 -- 172.21.15.84:0/1866347961 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80005510 con 0x7f3d78003bc0 2023-07-06T21:40:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d95dff700 1 -- 172.21.15.84:0/1866347961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 msgr2=0x7f3d78003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1866347961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d78003fb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001940 tx=0x7f3d78005880 comp rx=0 tx=0).stop 2023-07-06T21:40:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d8ffff700 1 -- 172.21.15.84:0/1866347961 reap_dead start 2023-07-06T21:40:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d95dff700 1 -- 172.21.15.84:0/1866347961 shutdown_connections 2023-07-06T21:40:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d95dff700 1 -- 172.21.15.84:0/1866347961 >> 172.21.15.84:0/1866347961 conn(0x7f3d9006b2b0 msgr2=0x7f3d9006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d95dff700 1 -- 172.21.15.84:0/1866347961 shutdown_connections 2023-07-06T21:40:16.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.281+0000 7f3d95dff700 1 -- 172.21.15.84:0/1866347961 wait complete. 2023-07-06T21:40:16.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.283+0000 7f3d95dff700 1 Processor -- start 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.283+0000 7f3d95dff700 1 -- start start 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d95dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d900cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d95dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9015ebf0 0x7f3d9015f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d95dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900aaee0 0x7f3d900ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d95dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900aab30 con 0x7f3d78003bc0 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d95dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9015f5a0 con 0x7f3d9015ebf0 2023-07-06T21:40:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d95dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900ab890 con 0x7f3d900aaee0 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d900cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d900cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43534/0 (socket says 172.21.15.84:43534) 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1907475867 learned_addr learned my addr 172.21.15.84:0/1907475867 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8effd700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9015ebf0 0x7f3d9015f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8ffff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900aaee0 0x7f3d900ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900aaee0 msgr2=0x7f3d900ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900aaee0 0x7f3d900ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9015ebf0 msgr2=0x7f3d9015f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9015ebf0 0x7f3d9015f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.284+0000 7f3d8f7fe700 1 -- 172.21.15.84:0/1907475867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900abb30 con 0x7f3d78003bc0 2023-07-06T21:40:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.285+0000 7f3d8f7fe700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d900cb890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c3c0 tx=0x7f3d800012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.285+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80006c70 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.285+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d80006dd0 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.285+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d900cadb0 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.285+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d80007280 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.285+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d900cb370 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.286+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d900cb590 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.287+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d800073e0 con 0x7f3d78003bc0 2023-07-06T21:40:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.288+0000 7f3d77fff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7c06ad10 0x7f3d7c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.288+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3d8008a480 con 0x7f3d78003bc0 2023-07-06T21:40:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.288+0000 7f3d8effd700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7c06ad10 0x7f3d7c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.289+0000 7f3d8effd700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7c06ad10 0x7f3d7c06d1c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d9015c880 tx=0x7f3d84009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.292+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3d800554e0 con 0x7f3d78003bc0 2023-07-06T21:40:16.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.488+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d9004e1d0 con 0x7f3d78003bc0 2023-07-06T21:40:16.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.488+0000 7f3d77fff700 1 -- 172.21.15.84:0/1907475867 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d80058b00 con 0x7f3d78003bc0 2023-07-06T21:40:16.491 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7c06ad10 msgr2=0x7f3d7c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7c06ad10 0x7f3d7c06d1c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d9015c880 tx=0x7f3d84009210 comp rx=0 tx=0).stop 2023-07-06T21:40:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 msgr2=0x7f3d900cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d900cb890 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3d8000c3c0 tx=0x7f3d800012d0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 shutdown_connections 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7c06ad10 0x7f3d7c06d1c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d900aaee0 0x7f3d900ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d78003bc0 0x7f3d900cb890 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 --2- 172.21.15.84:0/1907475867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d9015ebf0 0x7f3d9015f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 >> 172.21.15.84:0/1907475867 conn(0x7f3d9006b2b0 msgr2=0x7f3d900d3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 shutdown_connections 2023-07-06T21:40:16.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.491+0000 7f3d95dff700 1 -- 172.21.15.84:0/1907475867 wait complete. 2023-07-06T21:40:16.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:16.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.709+0000 7f32053ff700 1 Processor -- start 2023-07-06T21:40:16.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.709+0000 7f32053ff700 1 -- start start 2023-07-06T21:40:16.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f32053ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000b1a80 0x7f32000d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f32053ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32000d1830 con 0x7f32000b81c0 2023-07-06T21:40:16.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f32053ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32000cf330 con 0x7f32000cfd10 2023-07-06T21:40:16.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f32053ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32000d1270 con 0x7f32000b1a80 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f31feffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f32000b81c0 0x7f32000cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36324/0 (socket says 172.21.15.84:36324) 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f31feffd700 1 -- 172.21.15.84:0/2393240265 learned_addr learned my addr 172.21.15.84:0/2393240265 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.710+0000 7f31ff7fe700 1 --2- 172.21.15.84:0/2393240265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000b1a80 0x7f32000d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926252878 0 0) 0x7f32000d1830 con 0x7f32000b81c0 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31ec0035b0 con 0x7f32000b81c0 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540496791 0 0) 0x7f32000cf330 con 0x7f32000cfd10 2023-07-06T21:40:16.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32000d1830 con 0x7f32000cfd10 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31ff7fe700 1 -- 172.21.15.84:0/2393240265 >> v1:172.21.15.107:6789/0 conn(0x7f32000cfd10 legacy=0x7f32000d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258791476 0 0) 0x7f31ec0035b0 con 0x7f32000b81c0 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31ff7fe700 1 -- 172.21.15.84:0/2393240265 >> v1:172.21.15.84:6789/0 conn(0x7f32000b81c0 legacy=0x7f32000cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31ff7fe700 1 -- 172.21.15.84:0/2393240265 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32000d1550 con 0x7f32000b1a80 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.711+0000 7f31ff7fe700 1 --2- 172.21.15.84:0/2393240265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000b1a80 0x7f32000d2a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009dc0 tx=0x7f31f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6de53b0411a9d5a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800e070 con 0x7f32000b1a80 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f8003a20 con 0x7f32000b1a80 2023-07-06T21:40:16.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f31fe7fc700 1 -- 172.21.15.84:0/2393240265 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0001670 con 0x7f32000b1a80 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 -- 172.21.15.84:0/2393240265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000b1a80 msgr2=0x7f32000d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 --2- 172.21.15.84:0/2393240265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000b1a80 0x7f32000d2a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31f8009dc0 tx=0x7f31f800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 -- 172.21.15.84:0/2393240265 shutdown_connections 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 --2- 172.21.15.84:0/2393240265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000b1a80 0x7f32000d2a90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 -- 172.21.15.84:0/2393240265 >> 172.21.15.84:0/2393240265 conn(0x7f320006b2b0 msgr2=0x7f320006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 -- 172.21.15.84:0/2393240265 shutdown_connections 2023-07-06T21:40:16.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.712+0000 7f32053ff700 1 -- 172.21.15.84:0/2393240265 wait complete. 2023-07-06T21:40:16.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.714+0000 7f32053ff700 1 Processor -- start 2023-07-06T21:40:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.714+0000 7f32053ff700 1 -- start start 2023-07-06T21:40:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f32053ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32000b1a80 0x7f3200158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f32053ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 0x7f3200156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f32053ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000cfd10 0x7f32000c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f32053ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200158620 con 0x7f32000b81c0 2023-07-06T21:40:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f32053ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200157cc0 con 0x7f32000b1a80 2023-07-06T21:40:16.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f32053ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3200157f60 con 0x7f32000cfd10 2023-07-06T21:40:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f31f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 0x7f3200156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f31f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 0x7f3200156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43540/0 (socket says 172.21.15.84:43540) 2023-07-06T21:40:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f31f7fff700 1 -- 172.21.15.84:0/30582452 learned_addr learned my addr 172.21.15.84:0/30582452 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f31ff7fe700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000cfd10 0x7f32000c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.715+0000 7f31feffd700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32000b1a80 0x7f3200158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f7fff700 1 -- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000cfd10 msgr2=0x7f32000c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f7fff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000cfd10 0x7f32000c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f7fff700 1 -- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32000b1a80 msgr2=0x7f3200158a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f7fff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32000b1a80 0x7f3200158a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f7fff700 1 -- 172.21.15.84:0/30582452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009a70 con 0x7f32000b81c0 2023-07-06T21:40:16.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f7fff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 0x7f3200156830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c4d0 tx=0x7f31e800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80052d0 con 0x7f32000b81c0 2023-07-06T21:40:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e8005430 con 0x7f32000b81c0 2023-07-06T21:40:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3200158240 con 0x7f32000b81c0 2023-07-06T21:40:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8003880 con 0x7f32000b81c0 2023-07-06T21:40:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.716+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32000c9050 con 0x7f32000b81c0 2023-07-06T21:40:16.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.717+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3200158760 con 0x7f32000b81c0 2023-07-06T21:40:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.718+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31e80039e0 con 0x7f32000b81c0 2023-07-06T21:40:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.719+0000 7f31f77fe700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31e006acb0 0x7f31e006d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:16.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.719+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f31e80886a0 con 0x7f32000b81c0 2023-07-06T21:40:16.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.719+0000 7f31feffd700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31e006acb0 0x7f31e006d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:16.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.720+0000 7f31feffd700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31e006acb0 0x7f31e006d160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f31f0001380 tx=0x7f31f0001770 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:16.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.723+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3200158760 con 0x7f32000b81c0 2023-07-06T21:40:16.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.920+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3200158760 con 0x7f32000b81c0 2023-07-06T21:40:16.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.920+0000 7f31f77fe700 1 -- 172.21.15.84:0/30582452 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3200158760 con 0x7f32000b81c0 2023-07-06T21:40:16.923 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:16.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31e006acb0 msgr2=0x7f31e006d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31e006acb0 0x7f31e006d160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f31f0001380 tx=0x7f31f0001770 comp rx=0 tx=0).stop 2023-07-06T21:40:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 msgr2=0x7f3200156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 0x7f3200156830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c4d0 tx=0x7f31e800c7e0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 shutdown_connections 2023-07-06T21:40:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31e006acb0 0x7f31e006d160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32000cfd10 0x7f32000c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32000b1a80 0x7f3200158a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 --2- 172.21.15.84:0/30582452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32000b81c0 0x7f3200156830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.922+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 >> 172.21.15.84:0/30582452 conn(0x7f320006b2b0 msgr2=0x7f32001e7060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.923+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 shutdown_connections 2023-07-06T21:40:16.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:16.923+0000 7f32053ff700 1 -- 172.21.15.84:0/30582452 wait complete. 2023-07-06T21:40:16.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:17.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.139+0000 7f676c1ff700 1 Processor -- start 2023-07-06T21:40:17.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.139+0000 7f676c1ff700 1 -- start start 2023-07-06T21:40:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f676c1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f67640d1180 0x7f67640cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f676c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640cf9b0 0x7f6764165e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f676c1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67640cef90 con 0x7f676415c4f0 2023-07-06T21:40:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f676c1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67640d1740 con 0x7f67640d1180 2023-07-06T21:40:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f676c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640cd290 con 0x7f67640cf9b0 2023-07-06T21:40:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f6769f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f676415c4f0 0x7f67640b59d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36332/0 (socket says 172.21.15.84:36332) 2023-07-06T21:40:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.140+0000 7f676979a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f67640d1180 0x7f67640cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f6769f9b700 1 -- 172.21.15.84:0/3609142276 learned_addr learned my addr 172.21.15.84:0/3609142276 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676979a700 1 -- 172.21.15.84:0/3609142276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640cf9b0 msgr2=0x7f6764165e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676a79c700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640cf9b0 0x7f6764165e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676979a700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640cf9b0 0x7f6764165e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676979a700 1 -- 172.21.15.84:0/3609142276 >> v1:172.21.15.84:6789/0 conn(0x7f676415c4f0 legacy=0x7f67640b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676979a700 1 -- 172.21.15.84:0/3609142276 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640cd570 con 0x7f67640d1180 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676a79c700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640cf9b0 0x7f6764165e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:17.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f6768f99700 1 -- 172.21.15.84:0/3609142276 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163492320 0 0) 0x7f67640cef90 con 0x7f676415c4f0 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f676979a700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640d1180 0x7f67640cf470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6754009840 tx=0x7f6754009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae47ac000dbe77f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f6768f99700 1 -- 172.21.15.84:0/3609142276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754011830 con 0x7f67640d1180 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f6768f99700 1 -- 172.21.15.84:0/3609142276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6754011990 con 0x7f67640d1180 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.141+0000 7f6768f99700 1 -- 172.21.15.84:0/3609142276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754011d20 con 0x7f67640d1180 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 -- 172.21.15.84:0/3609142276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640d1180 msgr2=0x7f67640cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640d1180 0x7f67640cf470 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6754009840 tx=0x7f6754009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 -- 172.21.15.84:0/3609142276 shutdown_connections 2023-07-06T21:40:17.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640cf9b0 0x7f6764165e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 --2- 172.21.15.84:0/3609142276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640d1180 0x7f67640cf470 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 -- 172.21.15.84:0/3609142276 >> 172.21.15.84:0/3609142276 conn(0x7f676406b3c0 msgr2=0x7f676406b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 -- 172.21.15.84:0/3609142276 shutdown_connections 2023-07-06T21:40:17.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.142+0000 7f676c1ff700 1 -- 172.21.15.84:0/3609142276 wait complete. 2023-07-06T21:40:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.144+0000 7f676c1ff700 1 Processor -- start 2023-07-06T21:40:17.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.144+0000 7f676c1ff700 1 -- start start 2023-07-06T21:40:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676c1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 0x7f67640c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640d1180 0x7f67640b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676c1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67640b6e20 0x7f67641e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676c1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640b73a0 con 0x7f67640cf9b0 2023-07-06T21:40:17.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676c1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640b7680 con 0x7f67640b6e20 2023-07-06T21:40:17.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67640b7960 con 0x7f67640d1180 2023-07-06T21:40:17.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f6769f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 0x7f67640c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f6769f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 0x7f67640c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43562/0 (socket says 172.21.15.84:43562) 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f6769f9b700 1 -- 172.21.15.84:0/2705513639 learned_addr learned my addr 172.21.15.84:0/2705513639 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676979a700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640d1180 0x7f67640b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.145+0000 7f676a79c700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67640b6e20 0x7f67641e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f6769f9b700 1 -- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640d1180 msgr2=0x7f67640b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f6769f9b700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640d1180 0x7f67640b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f6769f9b700 1 -- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67640b6e20 msgr2=0x7f67641e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f6769f9b700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67640b6e20 0x7f67641e4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f6769f9b700 1 -- 172.21.15.84:0/2705513639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540094a0 con 0x7f67640cf9b0 2023-07-06T21:40:17.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f6769f9b700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 0x7f67640c8c50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f676000c550 tx=0x7f676000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:17.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760004c80 con 0x7f67640cf9b0 2023-07-06T21:40:17.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760004e00 con 0x7f67640cf9b0 2023-07-06T21:40:17.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67640b7c40 con 0x7f67640cf9b0 2023-07-06T21:40:17.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760007310 con 0x7f67640cf9b0 2023-07-06T21:40:17.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.146+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676415cb70 con 0x7f67640cf9b0 2023-07-06T21:40:17.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.147+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676404e1d0 con 0x7f67640cf9b0 2023-07-06T21:40:17.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.148+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6760007470 con 0x7f67640cf9b0 2023-07-06T21:40:17.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.149+0000 7f675a7fc700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674806ace0 0x7f674806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.149+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6760089230 con 0x7f67640cf9b0 2023-07-06T21:40:17.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.150+0000 7f676979a700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674806ace0 0x7f674806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.150+0000 7f676979a700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674806ace0 0x7f674806d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67540090b0 tx=0x7f6754008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:17.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.153+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f67600542a0 con 0x7f67640cf9b0 2023-07-06T21:40:17.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.350+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67640d24c0 con 0x7f67640cf9b0 2023-07-06T21:40:17.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.350+0000 7f675a7fc700 1 -- 172.21.15.84:0/2705513639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67600578c0 con 0x7f67640cf9b0 2023-07-06T21:40:17.353 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674806ace0 msgr2=0x7f674806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674806ace0 0x7f674806d190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67540090b0 tx=0x7f6754008030 comp rx=0 tx=0).stop 2023-07-06T21:40:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 msgr2=0x7f67640c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 0x7f67640c8c50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f676000c550 tx=0x7f676000c910 comp rx=0 tx=0).stop 2023-07-06T21:40:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 shutdown_connections 2023-07-06T21:40:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674806ace0 0x7f674806d190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67640b6e20 0x7f67641e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67640cf9b0 0x7f67640c8c50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 --2- 172.21.15.84:0/2705513639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67640d1180 0x7f67640b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 >> 172.21.15.84:0/2705513639 conn(0x7f676406b3c0 msgr2=0x7f6764158220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:17.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 shutdown_connections 2023-07-06T21:40:17.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.353+0000 7f676c1ff700 1 -- 172.21.15.84:0/2705513639 wait complete. 2023-07-06T21:40:17.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.568+0000 7f1cd31ff700 1 Processor -- start 2023-07-06T21:40:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.568+0000 7f1cd31ff700 1 -- start start 2023-07-06T21:40:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1cd31ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1ccc0cf280 0x7f1ccc0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1cd31ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc0c94b0 con 0x7f1ccc06dde0 2023-07-06T21:40:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1cd31ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc0ca9b0 con 0x7f1ccc069e90 2023-07-06T21:40:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1cd31ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc0cac90 con 0x7f1ccc0cf280 2023-07-06T21:40:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1ccbfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1ccc06dde0 0x7f1ccc06a260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36344/0 (socket says 172.21.15.84:36344) 2023-07-06T21:40:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1ccbfff700 1 -- 172.21.15.84:0/3002903971 learned_addr learned my addr 172.21.15.84:0/3002903971 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1cd179c700 1 --2- 172.21.15.84:0/3002903971 >> v2:172.21.15.107:3300/0 conn(0x7f1ccc0cf280 0x7f1ccc0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569867505 0 0) 0x7f1ccc0c94b0 con 0x7f1ccc06dde0 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.570+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cbc0035b0 con 0x7f1ccc06dde0 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346785189 0 0) 0x7f1ccc0ca9b0 con 0x7f1ccc069e90 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ccc0c94b0 con 0x7f1ccc069e90 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485133912 0 0) 0x7f1cbc0035b0 con 0x7f1ccc06dde0 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 >> v2:172.21.15.107:3300/0 conn(0x7f1ccc0cf280 msgr2=0x7f1ccc0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 --2- 172.21.15.84:0/3002903971 >> v2:172.21.15.107:3300/0 conn(0x7f1ccc0cf280 0x7f1ccc0b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 >> v1:172.21.15.107:6789/0 conn(0x7f1ccc069e90 legacy=0x7f1ccc06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0caf70 con 0x7f1ccc06dde0 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f1cb8001670 con 0x7f1ccc06dde0 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 >> v1:172.21.15.84:6789/0 conn(0x7f1ccc06dde0 legacy=0x7f1ccc06a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1cbc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cbc004850 0x7f1cbc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cbc005200 0x7f1cbc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0cac90 con 0x7f1cbc003b90 2023-07-06T21:40:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc00c110 con 0x7f1cbc005200 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc00c2d0 con 0x7f1cbc004850 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1cbc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1ccbfff700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cbc004850 0x7f1cbc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.571+0000 7f1cd179c700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cbc005200 0x7f1cbc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cbc004850 msgr2=0x7f1cbc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1cbc004850 0x7f1cbc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cbc005200 msgr2=0x7f1cbc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:17.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1cbc005200 0x7f1cbc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/3002903971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc0035b0 con 0x7f1cbc003b90 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd179c700 1 -- 172.21.15.84:0/3002903971 reap_dead start 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1cbc003f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0007fd0 tx=0x7f1cbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4bc38992f79993 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0001430 con 0x7f1cbc003b90 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc00052d0 con 0x7f1cbc003b90 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1ccb7fe700 1 -- 172.21.15.84:0/3002903971 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0001d40 con 0x7f1cbc003b90 2023-07-06T21:40:17.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd31ff700 1 -- 172.21.15.84:0/3002903971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 msgr2=0x7f1cbc003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.572+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1cbc003f80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0007fd0 tx=0x7f1cbc005880 comp rx=0 tx=0).stop 2023-07-06T21:40:17.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.573+0000 7f1cd31ff700 1 -- 172.21.15.84:0/3002903971 shutdown_connections 2023-07-06T21:40:17.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.573+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/3002903971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1cbc003f80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.573+0000 7f1cd31ff700 1 -- 172.21.15.84:0/3002903971 >> 172.21.15.84:0/3002903971 conn(0x7f1ccc0cbf10 msgr2=0x7f1ccc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:17.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.573+0000 7f1cd31ff700 1 -- 172.21.15.84:0/3002903971 shutdown_connections 2023-07-06T21:40:17.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.573+0000 7f1cd31ff700 1 -- 172.21.15.84:0/3002903971 wait complete. 2023-07-06T21:40:17.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.574+0000 7f1cd31ff700 1 Processor -- start 2023-07-06T21:40:17.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.575+0000 7f1cd31ff700 1 -- start start 2023-07-06T21:40:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.575+0000 7f1cd31ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1ccc152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.575+0000 7f1cd31ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ccc15ba40 0x7f1ccc15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.575+0000 7f1cd31ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ccc15ae90 0x7f1ccc15b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.575+0000 7f1cd31ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc153060 con 0x7f1cbc003b90 2023-07-06T21:40:17.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.575+0000 7f1cd31ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0ae9b0 con 0x7f1ccc15ae90 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd31ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc0aec90 con 0x7f1ccc15ba40 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1ccc152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1ccc152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43586/0 (socket says 172.21.15.84:43586) 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/2660651626 learned_addr learned my addr 172.21.15.84:0/2660651626 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1ccbfff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ccc15ba40 0x7f1ccc15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd179c700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ccc15ae90 0x7f1ccc15b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ccc15ba40 msgr2=0x7f1ccc15a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ccc15ba40 0x7f1ccc15a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ccc15ae90 msgr2=0x7f1ccc15b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ccc15ae90 0x7f1ccc15b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 -- 172.21.15.84:0/2660651626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0aef70 con 0x7f1cbc003b90 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd179c700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ccc15ae90 0x7f1ccc15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.576+0000 7f1cd0f9b700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1ccc152b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c0e0 tx=0x7f1cc00012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:17.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.577+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0001d40 con 0x7f1cbc003b90 2023-07-06T21:40:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.577+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc00013d0 con 0x7f1cbc003b90 2023-07-06T21:40:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.577+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc0af250 con 0x7f1cbc003b90 2023-07-06T21:40:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.577+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0007040 con 0x7f1cbc003b90 2023-07-06T21:40:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.577+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc15c940 con 0x7f1cbc003b90 2023-07-06T21:40:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.578+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc0af5c0 con 0x7f1cbc003b90 2023-07-06T21:40:17.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.579+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1cc00071a0 con 0x7f1cbc003b90 2023-07-06T21:40:17.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.580+0000 7f1cc97fa700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1cb006ace0 0x7f1cb006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:17.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.580+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1cc008a080 con 0x7f1cbc003b90 2023-07-06T21:40:17.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.580+0000 7f1ccbfff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1cb006ace0 0x7f1cb006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:17.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.580+0000 7f1ccbfff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1cb006ace0 0x7f1cb006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc15be10 tx=0x7f1cb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:17.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.584+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1cc00bc310 con 0x7f1cbc003b90 2023-07-06T21:40:17.754 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:40:17.759 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:40:17.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.785+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1ccc0c8eb0 con 0x7f1cbc003b90 2023-07-06T21:40:17.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.786+0000 7f1cc97fa700 1 -- 172.21.15.84:0/2660651626 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1cc0017020 con 0x7f1cbc003b90 2023-07-06T21:40:17.788 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:17.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1cb006ace0 msgr2=0x7f1cb006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1cb006ace0 0x7f1cb006d190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc15be10 tx=0x7f1cb8009040 comp rx=0 tx=0).stop 2023-07-06T21:40:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 msgr2=0x7f1ccc152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1ccc152b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00c0e0 tx=0x7f1cc00012d0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 shutdown_connections 2023-07-06T21:40:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1cb006ace0 0x7f1cb006d190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ccc15ae90 0x7f1ccc15b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1cbc003b90 0x7f1ccc152b20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 --2- 172.21.15.84:0/2660651626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ccc15ba40 0x7f1ccc15a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 >> 172.21.15.84:0/2660651626 conn(0x7f1ccc0cbf10 msgr2=0x7f1ccc0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.788+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 shutdown_connections 2023-07-06T21:40:17.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:17.789+0000 7f1cd31ff700 1 -- 172.21.15.84:0/2660651626 wait complete. 2023-07-06T21:40:17.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:18.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.041+0000 7f1e4c7ff700 1 Processor -- start 2023-07-06T21:40:18.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.042+0000 7f1e4c7ff700 1 -- start start 2023-07-06T21:40:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e4c7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 0x7f1e440cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e4c7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440d0a80 con 0x7f1e440d3710 2023-07-06T21:40:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e4c7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440b8790 con 0x7f1e440cfc60 2023-07-06T21:40:18.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e4c7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440b4e70 con 0x7f1e440b83c0 2023-07-06T21:40:18.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e49d9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f1e440d3710 0x7f1e440cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36346/0 (socket says 172.21.15.84:36346) 2023-07-06T21:40:18.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2896108753 learned_addr learned my addr 172.21.15.84:0/2896108753 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:18.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886970698 0 0) 0x7f1e440d0a80 con 0x7f1e440d3710 2023-07-06T21:40:18.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e4a59b700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 0x7f1e440cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.043+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e300035b0 con 0x7f1e440d3710 2023-07-06T21:40:18.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28247918 0 0) 0x7f1e300035b0 con 0x7f1e440d3710 2023-07-06T21:40:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 msgr2=0x7f1e440cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 0x7f1e440cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 >> v1:172.21.15.107:6789/0 conn(0x7f1e440cfc60 legacy=0x7f1e440d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440b5150 con 0x7f1e440d3710 2023-07-06T21:40:18.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f1e34001670 con 0x7f1e440d3710 2023-07-06T21:40:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 >> v1:172.21.15.84:6789/0 conn(0x7f1e440d3710 legacy=0x7f1e440cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e30003a30 0x7f1e30003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e30004850 0x7f1e30004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 0x7f1e300052b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440b4e70 con 0x7f1e30004850 2023-07-06T21:40:18.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3000bba0 con 0x7f1e30003a30 2023-07-06T21:40:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3000bd30 con 0x7f1e440b83c0 2023-07-06T21:40:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e30004850 0x7f1e30004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e4a59b700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e30003a30 0x7f1e30003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e4ad9c700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 0x7f1e300052b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 msgr2=0x7f1e300052b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:18.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e440b83c0 0x7f1e300052b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e30003a30 msgr2=0x7f1e30003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.044+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e30003a30 0x7f1e30003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2896108753 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300035b0 con 0x7f1e30004850 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e30004850 0x7f1e30004cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c160 tx=0x7f1e30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d526405b3000d814 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34007b70 con 0x7f1e30004850 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3400cce0 con 0x7f1e30004850 2023-07-06T21:40:18.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e49599700 1 -- 172.21.15.84:0/2896108753 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34005ba0 con 0x7f1e30004850 2023-07-06T21:40:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2896108753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e30004850 msgr2=0x7f1e30004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2896108753 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e30004850 0x7f1e30004cc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c160 tx=0x7f1e30005880 comp rx=0 tx=0).stop 2023-07-06T21:40:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.045+0000 7f1e4ad9c700 1 -- 172.21.15.84:0/2896108753 reap_dead start 2023-07-06T21:40:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.046+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2896108753 shutdown_connections 2023-07-06T21:40:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.046+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2896108753 >> 172.21.15.84:0/2896108753 conn(0x7f1e4406b350 msgr2=0x7f1e4406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:18.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.046+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2896108753 shutdown_connections 2023-07-06T21:40:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.046+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2896108753 wait complete. 2023-07-06T21:40:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.047+0000 7f1e4c7ff700 1 Processor -- start 2023-07-06T21:40:18.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.047+0000 7f1e4c7ff700 1 -- start start 2023-07-06T21:40:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.048+0000 7f1e4c7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e30004850 0x7f1e44154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.048+0000 7f1e4c7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 0x7f1e441540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.048+0000 7f1e4c7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e440d1860 0x7f1e440d1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.048+0000 7f1e4c7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440c6ff0 con 0x7f1e440c7970 2023-07-06T21:40:18.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.048+0000 7f1e4c7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440d2240 con 0x7f1e440d1860 2023-07-06T21:40:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.048+0000 7f1e4c7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440c7ff0 con 0x7f1e30004850 2023-07-06T21:40:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 0x7f1e441540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 0x7f1e441540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43596/0 (socket says 172.21.15.84:43596) 2023-07-06T21:40:18.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2481642709 learned_addr learned my addr 172.21.15.84:0/2481642709 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e4ad9c700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e440d1860 0x7f1e440d1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e4a59b700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e30004850 0x7f1e44154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e30004850 msgr2=0x7f1e44154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e30004850 0x7f1e44154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e440d1860 msgr2=0x7f1e440d1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e440d1860 0x7f1e440d1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 -- 172.21.15.84:0/2481642709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44155130 con 0x7f1e440c7970 2023-07-06T21:40:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e49d9a700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 0x7f1e441540d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c750 tx=0x7f1e34001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.049+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34004870 con 0x7f1e440c7970 2023-07-06T21:40:18.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.050+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e340049d0 con 0x7f1e440c7970 2023-07-06T21:40:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.050+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34004d80 con 0x7f1e440c7970 2023-07-06T21:40:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.050+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e44154610 con 0x7f1e440c7970 2023-07-06T21:40:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.050+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e441553b0 con 0x7f1e440c7970 2023-07-06T21:40:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.051+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e440c7130 con 0x7f1e440c7970 2023-07-06T21:40:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.052+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1e34019070 con 0x7f1e440c7970 2023-07-06T21:40:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.053+0000 7f1e3affd700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e2806ad10 0x7f1e2806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.053+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1e3408a680 con 0x7f1e440c7970 2023-07-06T21:40:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.053+0000 7f1e4a59b700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e2806ad10 0x7f1e2806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.054+0000 7f1e4a59b700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e2806ad10 0x7f1e2806d1c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009c60 tx=0x7f1e40009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:18.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.057+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1e440c7130 con 0x7f1e440c7970 2023-07-06T21:40:18.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.272+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e440c7130 con 0x7f1e440c7970 2023-07-06T21:40:18.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.272+0000 7f1e3affd700 1 -- 172.21.15.84:0/2481642709 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1e440c7130 con 0x7f1e440c7970 2023-07-06T21:40:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.274+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e2806ad10 msgr2=0x7f1e2806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.274+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e2806ad10 0x7f1e2806d1c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009c60 tx=0x7f1e40009210 comp rx=0 tx=0).stop 2023-07-06T21:40:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.274+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 msgr2=0x7f1e441540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.274+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 0x7f1e441540d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400c750 tx=0x7f1e34001930 comp rx=0 tx=0).stop 2023-07-06T21:40:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 shutdown_connections 2023-07-06T21:40:18.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1e2806ad10 0x7f1e2806d1c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e440d1860 0x7f1e440d1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1e30004850 0x7f1e44154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 --2- 172.21.15.84:0/2481642709 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e440c7970 0x7f1e441540d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 >> 172.21.15.84:0/2481642709 conn(0x7f1e4406b350 msgr2=0x7f1e4415aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:18.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 shutdown_connections 2023-07-06T21:40:18.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.275+0000 7f1e4c7ff700 1 -- 172.21.15.84:0/2481642709 wait complete. 2023-07-06T21:40:18.279 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:18.295 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:40:18.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:18.295 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:40:18.363 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:18.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:40:18.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:18.365 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:40:18.416 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:40:18.416 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:18.464 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:40:18.465 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:18.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:18.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.739+0000 7f6b6efff700 1 Processor -- start 2023-07-06T21:40:18.742 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.739+0000 7f6b6efff700 1 -- start start 2023-07-06T21:40:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.740+0000 7f6b6efff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b680cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.740+0000 7f6b6efff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6b6806a5a0 0x7f6b680c5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.740+0000 7f6b6efff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680c3f80 con 0x7f6b680c4d40 2023-07-06T21:40:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.740+0000 7f6b6efff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b680c4260 con 0x7f6b6806a5a0 2023-07-06T21:40:18.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.740+0000 7f6b6efff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680c4540 con 0x7f6b68158830 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b67fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6b680c4d40 0x7f6b68069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36354/0 (socket says 172.21.15.84:36354) 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b67fff700 1 -- 172.21.15.84:0/2629495662 learned_addr learned my addr 172.21.15.84:0/2629495662 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6d59c700 1 --2- 172.21.15.84:0/2629495662 >> v2:172.21.15.84:3300/0 conn(0x7f6b6806a5a0 0x7f6b680c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6cd9b700 1 --2- 172.21.15.84:0/2629495662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b680cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b677fe700 1 -- 172.21.15.84:0/2629495662 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631309826 0 0) 0x7f6b680c3f80 con 0x7f6b680c4d40 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6d59c700 1 -- 172.21.15.84:0/2629495662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 msgr2=0x7f6b680cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6d59c700 1 --2- 172.21.15.84:0/2629495662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b680cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6d59c700 1 -- 172.21.15.84:0/2629495662 >> v1:172.21.15.84:6789/0 conn(0x7f6b680c4d40 legacy=0x7f6b68069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6d59c700 1 -- 172.21.15.84:0/2629495662 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680c4820 con 0x7f6b6806a5a0 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.741+0000 7f6b6d59c700 1 --2- 172.21.15.84:0/2629495662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b6806a5a0 0x7f6b680c5190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009d90 tx=0x7f6b6000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85863b3e49202e6d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b677fe700 1 -- 172.21.15.84:0/2629495662 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6000bdd0 con 0x7f6b6806a5a0 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b677fe700 1 -- 172.21.15.84:0/2629495662 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60011460 con 0x7f6b6806a5a0 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b677fe700 1 -- 172.21.15.84:0/2629495662 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b60011810 con 0x7f6b6806a5a0 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 -- 172.21.15.84:0/2629495662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b6806a5a0 msgr2=0x7f6b680c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 --2- 172.21.15.84:0/2629495662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b6806a5a0 0x7f6b680c5190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009d90 tx=0x7f6b6000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 -- 172.21.15.84:0/2629495662 shutdown_connections 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 --2- 172.21.15.84:0/2629495662 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b6806a5a0 0x7f6b680c5190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 --2- 172.21.15.84:0/2629495662 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b680cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 -- 172.21.15.84:0/2629495662 >> 172.21.15.84:0/2629495662 conn(0x7f6b680cf3d0 msgr2=0x7f6b680cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 -- 172.21.15.84:0/2629495662 shutdown_connections 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.742+0000 7f6b6efff700 1 -- 172.21.15.84:0/2629495662 wait complete. 2023-07-06T21:40:18.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.744+0000 7f6b6efff700 1 Processor -- start 2023-07-06T21:40:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.744+0000 7f6b6efff700 1 -- start start 2023-07-06T21:40:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6efff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6806a5a0 0x7f6b6814d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6efff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 0x7f6b6815b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6efff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b6815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6efff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680a81c0 con 0x7f6b680c4d40 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6efff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680a84a0 con 0x7f6b6806a5a0 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6efff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68158f00 con 0x7f6b68158830 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b67fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 0x7f6b6815b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b67fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 0x7f6b6815b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43616/0 (socket says 172.21.15.84:43616) 2023-07-06T21:40:18.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b67fff700 1 -- 172.21.15.84:0/3450394061 learned_addr learned my addr 172.21.15.84:0/3450394061 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6d59c700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b6815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.745+0000 7f6b6cd9b700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6806a5a0 0x7f6b6814d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b67fff700 1 -- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 msgr2=0x7f6b6815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b67fff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b6815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b67fff700 1 -- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6806a5a0 msgr2=0x7f6b6814d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b67fff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6806a5a0 0x7f6b6814d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b67fff700 1 -- 172.21.15.84:0/3450394061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b60009a70 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b67fff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 0x7f6b6815b390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b540011e0 tx=0x7f6b5400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004c80 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b54004e00 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54007310 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b681591e0 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.746+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b681ed860 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.747+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b680a86c0 con 0x7f6b680c4d40 2023-07-06T21:40:18.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.748+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b54007470 con 0x7f6b680c4d40 2023-07-06T21:40:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.749+0000 7f6b657fa700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b4c06ace0 0x7f6b4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.749+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6b54089060 con 0x7f6b680c4d40 2023-07-06T21:40:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.750+0000 7f6b6cd9b700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b4c06ace0 0x7f6b4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:18.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.750+0000 7f6b6cd9b700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b4c06ace0 0x7f6b4c06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006b20 tx=0x7f6b5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:18.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.753+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b54054160 con 0x7f6b680c4d40 2023-07-06T21:40:18.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.954+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b681593d0 con 0x7f6b680c4d40 2023-07-06T21:40:18.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.955+0000 7f6b657fa700 1 -- 172.21.15.84:0/3450394061 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f6b54057780 con 0x7f6b680c4d40 2023-07-06T21:40:18.957 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:18.957 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:18.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b4c06ace0 msgr2=0x7f6b4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b4c06ace0 0x7f6b4c06d190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c006b20 tx=0x7f6b5c008040 comp rx=0 tx=0).stop 2023-07-06T21:40:18.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 msgr2=0x7f6b6815b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:18.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 0x7f6b6815b390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b540011e0 tx=0x7f6b5400ca30 comp rx=0 tx=0).stop 2023-07-06T21:40:18.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 shutdown_connections 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b4c06ace0 0x7f6b4c06d190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b68158830 0x7f6b6815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.957+0000 7f6b6efff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6806a5a0 0x7f6b6814d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.958+0000 7f6b6efff700 1 --2- 172.21.15.84:0/3450394061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b680c4d40 0x7f6b6815b390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.958+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 >> 172.21.15.84:0/3450394061 conn(0x7f6b680cf3d0 msgr2=0x7f6b68157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.958+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 shutdown_connections 2023-07-06T21:40:18.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:18.958+0000 7f6b6efff700 1 -- 172.21.15.84:0/3450394061 wait complete. 2023-07-06T21:40:18.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:18.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:19.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.176+0000 7f91ee1ff700 1 Processor -- start 2023-07-06T21:40:19.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.177+0000 7f91ee1ff700 1 -- start start 2023-07-06T21:40:19.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91ee1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f91e80b81c0 0x7f91e80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91ee1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e80d1830 con 0x7f91e80ceda0 2023-07-06T21:40:19.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91ee1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e8160d40 con 0x7f91e80b1a80 2023-07-06T21:40:19.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91ee1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e80cf330 con 0x7f91e80b81c0 2023-07-06T21:40:19.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91e7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f91e80ceda0 0x7f91e8160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36368/0 (socket says 172.21.15.84:36368) 2023-07-06T21:40:19.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91e7fff700 1 -- 172.21.15.84:0/1771420408 learned_addr learned my addr 172.21.15.84:0/1771420408 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:19.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1771420408 >> v2:172.21.15.107:3300/0 conn(0x7f91e80b81c0 0x7f91e80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.178+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287835607 0 0) 0x7f91e80d1830 con 0x7f91e80ceda0 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d00035b0 con 0x7f91e80ceda0 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1439287788 0 0) 0x7f91d00035b0 con 0x7f91e80ceda0 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 >> v2:172.21.15.107:3300/0 conn(0x7f91e80b81c0 msgr2=0x7f91e80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 --2- 172.21.15.84:0/1771420408 >> v2:172.21.15.107:3300/0 conn(0x7f91e80b81c0 0x7f91e80cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 >> v1:172.21.15.107:6789/0 conn(0x7f91e80b1a80 legacy=0x7f91e815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e80d1270 con 0x7f91e80ceda0 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f91e00019f0 con 0x7f91e80ceda0 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 >> v1:172.21.15.84:6789/0 conn(0x7f91e80ceda0 legacy=0x7f91e8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0003a00 0x7f91d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d0004850 0x7f91d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0005200 0x7f91d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e80cf330 con 0x7f91d0003a00 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d000c080 con 0x7f91d0004850 2023-07-06T21:40:19.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d000c240 con 0x7f91d0005200 2023-07-06T21:40:19.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.179+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0003a00 0x7f91d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e6ffd700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d0004850 0x7f91d0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e7fff700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0005200 0x7f91d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e77fe700 1 -- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0005200 msgr2=0x7f91d000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0005200 0x7f91d000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e77fe700 1 -- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d0004850 msgr2=0x7f91d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91d0004850 0x7f91d0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e77fe700 1 -- 172.21.15.84:0/1771420408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00035b0 con 0x7f91d0003a00 2023-07-06T21:40:19.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e7fff700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0005200 0x7f91d000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e7fff700 1 -- 172.21.15.84:0/1771420408 reap_dead start 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0003a00 0x7f91d0003e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009b60 tx=0x7f91d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41b224692655586 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d800e6c0 con 0x7f91d0003a00 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d800e820 con 0x7f91d0003a00 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.180+0000 7f91e67fc700 1 -- 172.21.15.84:0/1771420408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d800eb90 con 0x7f91d0003a00 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1771420408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0003a00 msgr2=0x7f91d0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0003a00 0x7f91d0003e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009b60 tx=0x7f91d0005880 comp rx=0 tx=0).stop 2023-07-06T21:40:19.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1771420408 shutdown_connections 2023-07-06T21:40:19.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1771420408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91d0003a00 0x7f91d0003e10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1771420408 >> 172.21.15.84:0/1771420408 conn(0x7f91e806b2b0 msgr2=0x7f91e806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:19.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1771420408 shutdown_connections 2023-07-06T21:40:19.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.181+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1771420408 wait complete. 2023-07-06T21:40:19.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.183+0000 7f91ee1ff700 1 Processor -- start 2023-07-06T21:40:19.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.183+0000 7f91ee1ff700 1 -- start start 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91ee1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0003a00 0x7f91e815c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91ee1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e81f5ca0 0x7f91e8159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91ee1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 0x7f91e80cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91ee1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e8159e40 con 0x7f91e81599d0 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91ee1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e8155ab0 con 0x7f91e81f5ca0 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91ee1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e815cf30 con 0x7f91d0003a00 2023-07-06T21:40:19.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 0x7f91e80cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 0x7f91e80cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43632/0 (socket says 172.21.15.84:43632) 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 -- 172.21.15.84:0/1804874024 learned_addr learned my addr 172.21.15.84:0/1804874024 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0003a00 0x7f91e815c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e6ffd700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e81f5ca0 0x7f91e8159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 -- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0003a00 msgr2=0x7f91e815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0003a00 0x7f91e815c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 -- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e81f5ca0 msgr2=0x7f91e8159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e81f5ca0 0x7f91e8159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e7fff700 1 -- 172.21.15.84:0/1804874024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e8154ec0 con 0x7f91e81599d0 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e6ffd700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e81f5ca0 0x7f91e8159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.184+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0003a00 0x7f91e815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.185+0000 7f91e7fff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 0x7f91e80cb6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91e0001580 tx=0x7f91e000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.185+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e000bb20 con 0x7f91e81599d0 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.185+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e000bca0 con 0x7f91e81599d0 2023-07-06T21:40:19.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.185+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0007030 con 0x7f91e81599d0 2023-07-06T21:40:19.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.185+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e8155180 con 0x7f91e81599d0 2023-07-06T21:40:19.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.185+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e8155740 con 0x7f91e81599d0 2023-07-06T21:40:19.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.186+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e8155c20 con 0x7f91e81599d0 2023-07-06T21:40:19.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.187+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f91e0007190 con 0x7f91e81599d0 2023-07-06T21:40:19.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.188+0000 7f91cffff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.188+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f91e00891b0 con 0x7f91e81599d0 2023-07-06T21:40:19.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.188+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.189+0000 7f91e77fe700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91d406ace0 0x7f91d406d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91d80090b0 tx=0x7f91d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:19.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.193+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f91e00542b0 con 0x7f91e81599d0 2023-07-06T21:40:19.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.387+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91e804e1d0 con 0x7f91e81599d0 2023-07-06T21:40:19.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.387+0000 7f91cffff700 1 -- 172.21.15.84:0/1804874024 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91e00190a0 con 0x7f91e81599d0 2023-07-06T21:40:19.390 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:19.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91d406ace0 msgr2=0x7f91d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91d406ace0 0x7f91d406d190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f91d80090b0 tx=0x7f91d8008030 comp rx=0 tx=0).stop 2023-07-06T21:40:19.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 msgr2=0x7f91e80cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 0x7f91e80cb6a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91e0001580 tx=0x7f91e000b0d0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 shutdown_connections 2023-07-06T21:40:19.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f91d406ace0 0x7f91d406d190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91e81599d0 0x7f91e80cb6a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91d0003a00 0x7f91e815c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 --2- 172.21.15.84:0/1804874024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e81f5ca0 0x7f91e8159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.389+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 >> 172.21.15.84:0/1804874024 conn(0x7f91e806b2b0 msgr2=0x7f91e815b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:19.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.390+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 shutdown_connections 2023-07-06T21:40:19.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.390+0000 7f91ee1ff700 1 -- 172.21.15.84:0/1804874024 wait complete. 2023-07-06T21:40:19.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.603+0000 7f85fddff700 1 Processor -- start 2023-07-06T21:40:19.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.604+0000 7f85fddff700 1 -- start start 2023-07-06T21:40:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85fddff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 0x7f85f80d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85fddff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85f80b1a80 0x7f85f80d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85fddff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f80d1830 con 0x7f85f80b81c0 2023-07-06T21:40:19.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85fddff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f80cf330 con 0x7f85f80b1a80 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85fddff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80d1270 con 0x7f85f80cfd10 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85f77fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f85f80b81c0 0x7f85f80cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36372/0 (socket says 172.21.15.84:36372) 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85f77fe700 1 -- 172.21.15.84:0/882027241 learned_addr learned my addr 172.21.15.84:0/882027241 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85effff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 0x7f85f80d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85f7fff700 1 --2- 172.21.15.84:0/882027241 >> v2:172.21.15.107:3300/0 conn(0x7f85f80b1a80 0x7f85f80d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.605+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713045322 0 0) 0x7f85f80d1830 con 0x7f85f80b81c0 2023-07-06T21:40:19.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e40035b0 con 0x7f85f80b81c0 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1659550863 0 0) 0x7f85e40035b0 con 0x7f85f80b81c0 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 msgr2=0x7f85f80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 0x7f85f80d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 >> v2:172.21.15.107:3300/0 conn(0x7f85f80b1a80 msgr2=0x7f85f80d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 --2- 172.21.15.84:0/882027241 >> v2:172.21.15.107:3300/0 conn(0x7f85f80b1a80 0x7f85f80d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80d1550 con 0x7f85f80b81c0 2023-07-06T21:40:19.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f85e8001670 con 0x7f85f80b81c0 2023-07-06T21:40:19.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 >> v1:172.21.15.84:6789/0 conn(0x7f85f80b81c0 legacy=0x7f85f80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85e4003a00 0x7f85e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85e40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 0x7f85e4004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80cf330 con 0x7f85e4004430 2023-07-06T21:40:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80d1270 con 0x7f85e4003a00 2023-07-06T21:40:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85e400ba10 con 0x7f85f80cfd10 2023-07-06T21:40:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f7fff700 1 --2- 172.21.15.84:0/882027241 >> v2:172.21.15.107:3300/0 conn(0x7f85f80b1a80 0x7f85f80d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:19.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85effff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85e40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.606+0000 7f85f77fe700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85e4003a00 0x7f85e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85f7fff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 0x7f85e4004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85effff700 1 -- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 msgr2=0x7f85e4004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85effff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80cfd10 0x7f85e4004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85effff700 1 -- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85e4003a00 msgr2=0x7f85e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:19.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85effff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85e4003a00 0x7f85e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85effff700 1 -- 172.21.15.84:0/882027241 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e40035b0 con 0x7f85e4004430 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85effff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85e40048f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009b30 tx=0x7f85e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ad5b4d6dd7639e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000e6c0 con 0x7f85e4004430 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e000e820 con 0x7f85e4004430 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.607+0000 7f85f6ffd700 1 -- 172.21.15.84:0/882027241 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e000eb90 con 0x7f85e4004430 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85fddff700 1 -- 172.21.15.84:0/882027241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 msgr2=0x7f85e40048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85fddff700 1 --2- 172.21.15.84:0/882027241 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85e40048f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009b30 tx=0x7f85e4005880 comp rx=0 tx=0).stop 2023-07-06T21:40:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85f7fff700 1 -- 172.21.15.84:0/882027241 reap_dead start 2023-07-06T21:40:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85fddff700 1 -- 172.21.15.84:0/882027241 shutdown_connections 2023-07-06T21:40:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85fddff700 1 -- 172.21.15.84:0/882027241 >> 172.21.15.84:0/882027241 conn(0x7f85f806b2b0 msgr2=0x7f85f806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85fddff700 1 -- 172.21.15.84:0/882027241 shutdown_connections 2023-07-06T21:40:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.608+0000 7f85fddff700 1 -- 172.21.15.84:0/882027241 wait complete. 2023-07-06T21:40:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.610+0000 7f85fddff700 1 Processor -- start 2023-07-06T21:40:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.610+0000 7f85fddff700 1 -- start start 2023-07-06T21:40:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85fddff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85f8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85fddff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80c8e50 0x7f85f8156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.620 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85fddff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f80c92c0 0x7f85f815e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85fddff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8152650 con 0x7f85e4004430 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85fddff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f8158ee0 con 0x7f85f80c92c0 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85fddff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80c7ad0 con 0x7f85f80c8e50 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85f8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85f8152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43658/0 (socket says 172.21.15.84:43658) 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 -- 172.21.15.84:0/2237950072 learned_addr learned my addr 172.21.15.84:0/2237950072 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:19.621 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85effff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80c8e50 0x7f85f8156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f7fff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f80c92c0 0x7f85f815e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 -- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80c8e50 msgr2=0x7f85f8156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80c8e50 0x7f85f8156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 -- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f80c92c0 msgr2=0x7f85f815e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f80c92c0 0x7f85f815e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.611+0000 7f85f77fe700 1 -- 172.21.15.84:0/2237950072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80c7db0 con 0x7f85e4004430 2023-07-06T21:40:19.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.612+0000 7f85f77fe700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85f8152110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f85e8007290 tx=0x7f85e8007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.612+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e80014c0 con 0x7f85e4004430 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.612+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e800f850 con 0x7f85e4004430 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.612+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f80c8090 con 0x7f85e4004430 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.612+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e800fc00 con 0x7f85e4004430 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.612+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f80c8650 con 0x7f85e4004430 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.613+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f80b1f20 con 0x7f85e4004430 2023-07-06T21:40:19.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.614+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f85e800fd60 con 0x7f85e4004430 2023-07-06T21:40:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.615+0000 7f85f4ff9700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85d806ace0 0x7f85d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.615+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f85e8089590 con 0x7f85e4004430 2023-07-06T21:40:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.615+0000 7f85effff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85d806ace0 0x7f85d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.616+0000 7f85effff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85d806ace0 0x7f85d806d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85e00090b0 tx=0x7f85e0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:19.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.619+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85e8054560 con 0x7f85e4004430 2023-07-06T21:40:19.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.813+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85f804e1d0 con 0x7f85e4004430 2023-07-06T21:40:19.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.813+0000 7f85f4ff9700 1 -- 172.21.15.84:0/2237950072 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85e8057b80 con 0x7f85e4004430 2023-07-06T21:40:19.816 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85d806ace0 msgr2=0x7f85d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85d806ace0 0x7f85d806d190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f85e00090b0 tx=0x7f85e0008030 comp rx=0 tx=0).stop 2023-07-06T21:40:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 msgr2=0x7f85f8152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85f8152110 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f85e8007290 tx=0x7f85e8007650 comp rx=0 tx=0).stop 2023-07-06T21:40:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 shutdown_connections 2023-07-06T21:40:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f85d806ace0 0x7f85d806d190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85f80c92c0 0x7f85f815e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85e4004430 0x7f85f8152110 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 --2- 172.21.15.84:0/2237950072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85f80c8e50 0x7f85f8156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 >> 172.21.15.84:0/2237950072 conn(0x7f85f806b2b0 msgr2=0x7f85f80d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 shutdown_connections 2023-07-06T21:40:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:19.816+0000 7f85fddff700 1 -- 172.21.15.84:0/2237950072 wait complete. 2023-07-06T21:40:19.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:20.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.030+0000 7f3d8a7ff700 1 Processor -- start 2023-07-06T21:40:20.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.030+0000 7f3d8a7ff700 1 -- start start 2023-07-06T21:40:20.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d8a7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d8406dde0 0x7f3d8406a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d8a7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3d840cf280 0x7f3d840b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d8a7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d840c94b0 con 0x7f3d84069e90 2023-07-06T21:40:20.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d8a7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d8406a7a0 con 0x7f3d8406dde0 2023-07-06T21:40:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d8a7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d840ca9b0 con 0x7f3d840cf280 2023-07-06T21:40:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d837fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d8406dde0 0x7f3d8406a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d837fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3d8406dde0 0x7f3d8406a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43674/0 (socket says 172.21.15.84:43674) 2023-07-06T21:40:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d837fe700 1 -- 172.21.15.84:0/1797840393 learned_addr learned my addr 172.21.15.84:0/1797840393 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d88d9c700 1 --2- 172.21.15.84:0/1797840393 >> v2:172.21.15.107:3300/0 conn(0x7f3d840cf280 0x7f3d840b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d837fe700 1 -- 172.21.15.84:0/1797840393 >> v2:172.21.15.107:3300/0 conn(0x7f3d840cf280 msgr2=0x7f3d840b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:20.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d837fe700 1 --2- 172.21.15.84:0/1797840393 >> v2:172.21.15.107:3300/0 conn(0x7f3d840cf280 0x7f3d840b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.032+0000 7f3d837fe700 1 -- 172.21.15.84:0/1797840393 >> v1:172.21.15.107:6789/0 conn(0x7f3d84069e90 legacy=0x7f3d8406ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d837fe700 1 -- 172.21.15.84:0/1797840393 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840cac90 con 0x7f3d8406dde0 2023-07-06T21:40:20.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d837fe700 1 --2- 172.21.15.84:0/1797840393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d8406dde0 0x7f3d8406a260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d840ce6f0 tx=0x7f3d78009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e17f2d3242e75fe4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d82ffd700 1 -- 172.21.15.84:0/1797840393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78011830 con 0x7f3d8406dde0 2023-07-06T21:40:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d82ffd700 1 -- 172.21.15.84:0/1797840393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78011990 con 0x7f3d8406dde0 2023-07-06T21:40:20.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d82ffd700 1 -- 172.21.15.84:0/1797840393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78011d20 con 0x7f3d8406dde0 2023-07-06T21:40:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1797840393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d8406dde0 msgr2=0x7f3d8406a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.033+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1797840393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d8406dde0 0x7f3d8406a260 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d840ce6f0 tx=0x7f3d78009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.034+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1797840393 shutdown_connections 2023-07-06T21:40:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.034+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1797840393 >> v2:172.21.15.107:3300/0 conn(0x7f3d840cf280 0x7f3d840b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.034+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1797840393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d8406dde0 0x7f3d8406a260 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.034+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1797840393 >> 172.21.15.84:0/1797840393 conn(0x7f3d840cbf10 msgr2=0x7f3d840ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.034+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1797840393 shutdown_connections 2023-07-06T21:40:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.034+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1797840393 wait complete. 2023-07-06T21:40:20.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.035+0000 7f3d8a7ff700 1 Processor -- start 2023-07-06T21:40:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 -- start start 2023-07-06T21:40:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d84069e90 0x7f3d840c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8406dde0 0x7f3d840c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 0x7f3d84152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840c6ca0 con 0x7f3d840cf280 2023-07-06T21:40:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840c5e10 con 0x7f3d8406dde0 2023-07-06T21:40:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.036+0000 7f3d8a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840c6090 con 0x7f3d84069e90 2023-07-06T21:40:20.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 0x7f3d84152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 0x7f3d84152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43688/0 (socket says 172.21.15.84:43688) 2023-07-06T21:40:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 -- 172.21.15.84:0/1304730024 learned_addr learned my addr 172.21.15.84:0/1304730024 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:20.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d837fe700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8406dde0 0x7f3d840c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d83fff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d84069e90 0x7f3d840c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 -- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d84069e90 msgr2=0x7f3d840c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d84069e90 0x7f3d840c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 -- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8406dde0 msgr2=0x7f3d840c58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8406dde0 0x7f3d840c58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 -- 172.21.15.84:0/1304730024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d78009550 con 0x7f3d840cf280 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d83fff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d84069e90 0x7f3d840c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.037+0000 7f3d88d9c700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 0x7f3d84152a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00efc0 tx=0x7f3d7c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.038+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c009ea0 con 0x7f3d840cf280 2023-07-06T21:40:20.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.038+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c00f040 con 0x7f3d840cf280 2023-07-06T21:40:20.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.038+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c014480 con 0x7f3d840cf280 2023-07-06T21:40:20.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.038+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d840c6370 con 0x7f3d840cf280 2023-07-06T21:40:20.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.038+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d840c6930 con 0x7f3d840cf280 2023-07-06T21:40:20.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.039+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d840c6e10 con 0x7f3d840cf280 2023-07-06T21:40:20.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.040+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d7c0145e0 con 0x7f3d840cf280 2023-07-06T21:40:20.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.040+0000 7f3d80ff9700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7006ace0 0x7f3d7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.041+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3d7c089df0 con 0x7f3d840cf280 2023-07-06T21:40:20.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.041+0000 7f3d83fff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7006ace0 0x7f3d7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.041+0000 7f3d83fff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7006ace0 0x7f3d7006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3d841f37e0 tx=0x7f3d74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.045+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3d7c054e60 con 0x7f3d840cf280 2023-07-06T21:40:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.243+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d8404e1d0 con 0x7f3d840cf280 2023-07-06T21:40:20.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.243+0000 7f3d80ff9700 1 -- 172.21.15.84:0/1304730024 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d7c018020 con 0x7f3d840cf280 2023-07-06T21:40:20.246 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7006ace0 msgr2=0x7f3d7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7006ace0 0x7f3d7006d190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3d841f37e0 tx=0x7f3d74009040 comp rx=0 tx=0).stop 2023-07-06T21:40:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 msgr2=0x7f3d84152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 0x7f3d84152a80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c00efc0 tx=0x7f3d7c00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 shutdown_connections 2023-07-06T21:40:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d7006ace0 0x7f3d7006d190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d840cf280 0x7f3d84152a80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d84069e90 0x7f3d840c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 --2- 172.21.15.84:0/1304730024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d8406dde0 0x7f3d840c58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 >> 172.21.15.84:0/1304730024 conn(0x7f3d840cbf10 msgr2=0x7f3d840b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 shutdown_connections 2023-07-06T21:40:20.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.246+0000 7f3d8a7ff700 1 -- 172.21.15.84:0/1304730024 wait complete. 2023-07-06T21:40:20.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.460+0000 7fdaa09ff700 1 Processor -- start 2023-07-06T21:40:20.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.460+0000 7fdaa09ff700 1 -- start start 2023-07-06T21:40:20.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.461+0000 7fdaa09ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0b81c0 0x7fda9c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.461+0000 7fdaa09ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda9c0b1a80 0x7fda9c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.461+0000 7fdaa09ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c0d1830 con 0x7fda9c0ceda0 2023-07-06T21:40:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.461+0000 7fdaa09ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c0cf330 con 0x7fda9c0b1a80 2023-07-06T21:40:20.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.461+0000 7fdaa09ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0d1270 con 0x7fda9c0b81c0 2023-07-06T21:40:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda99d9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda9c0b1a80 0x7fda9c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda9a59c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0b81c0 0x7fda9c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda9a59c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0b81c0 0x7fda9c0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:48656/0 (socket says 172.21.15.84:48656) 2023-07-06T21:40:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda99d9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda9c0b1a80 0x7fda9c0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:55332/0 (socket says 172.21.15.84:55332) 2023-07-06T21:40:20.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda9a59c700 1 -- 172.21.15.84:0/3684854600 learned_addr learned my addr 172.21.15.84:0/3684854600 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda99d9b700 1 -- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0b81c0 msgr2=0x7fda9c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda99d9b700 1 --2- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0b81c0 0x7fda9c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda99d9b700 1 -- 172.21.15.84:0/3684854600 >> v1:172.21.15.107:6789/0 conn(0x7fda9c0ceda0 legacy=0x7fda9c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.462+0000 7fda99d9b700 1 -- 172.21.15.84:0/3684854600 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0d1550 con 0x7fda9c0b1a80 2023-07-06T21:40:20.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fda99d9b700 1 --2- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b1a80 0x7fda9c0cfb40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda90009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc678e6cb4277c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fda9959a700 1 -- 172.21.15.84:0/3684854600 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011830 con 0x7fda9c0b1a80 2023-07-06T21:40:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fda9959a700 1 -- 172.21.15.84:0/3684854600 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90011990 con 0x7fda9c0b1a80 2023-07-06T21:40:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fda9959a700 1 -- 172.21.15.84:0/3684854600 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011d20 con 0x7fda9c0b1a80 2023-07-06T21:40:20.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fdaa09ff700 1 -- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b1a80 msgr2=0x7fda9c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b1a80 0x7fda9c0cfb40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda90009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fdaa09ff700 1 -- 172.21.15.84:0/3684854600 shutdown_connections 2023-07-06T21:40:20.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0b81c0 0x7fda9c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/3684854600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b1a80 0x7fda9c0cfb40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.463+0000 7fdaa09ff700 1 -- 172.21.15.84:0/3684854600 >> 172.21.15.84:0/3684854600 conn(0x7fda9c06b2b0 msgr2=0x7fda9c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.464+0000 7fdaa09ff700 1 -- 172.21.15.84:0/3684854600 shutdown_connections 2023-07-06T21:40:20.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.464+0000 7fdaa09ff700 1 -- 172.21.15.84:0/3684854600 wait complete. 2023-07-06T21:40:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.465+0000 7fdaa09ff700 1 Processor -- start 2023-07-06T21:40:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.465+0000 7fdaa09ff700 1 -- start start 2023-07-06T21:40:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fdaa09ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 0x7fda9c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fdaa09ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b81c0 0x7fda9c0b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fdaa09ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0ceda0 0x7fda9c1e3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fdaa09ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c15afb0 con 0x7fda9c0b1a80 2023-07-06T21:40:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fdaa09ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0c8f30 con 0x7fda9c0b81c0 2023-07-06T21:40:20.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fdaa09ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c15b400 con 0x7fda9c0ceda0 2023-07-06T21:40:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.466+0000 7fda9a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 0x7fda9c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 0x7fda9c15aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43704/0 (socket says 172.21.15.84:43704) 2023-07-06T21:40:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 -- 172.21.15.84:0/493937794 learned_addr learned my addr 172.21.15.84:0/493937794 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:20.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda99d9b700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b81c0 0x7fda9c0b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 -- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0ceda0 msgr2=0x7fda9c1e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9ad9d700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0ceda0 0x7fda9c1e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0ceda0 0x7fda9c1e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 -- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b81c0 msgr2=0x7fda9c0b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b81c0 0x7fda9c0b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 -- 172.21.15.84:0/493937794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900094a0 con 0x7fda9c0b1a80 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda99d9b700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b81c0 0x7fda9c0b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9ad9d700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0ceda0 0x7fda9c1e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda9a59c700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 0x7fda9c15aa70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e910 tx=0x7fda8c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c0096a0 con 0x7fda9c0b1a80 2023-07-06T21:40:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c00c850 con 0x7fda9c0b1a80 2023-07-06T21:40:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c00cc00 con 0x7fda9c0b1a80 2023-07-06T21:40:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.467+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c0c8320 con 0x7fda9c0b1a80 2023-07-06T21:40:20.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.468+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c0c88e0 con 0x7fda9c0b1a80 2023-07-06T21:40:20.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.469+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c0c9210 con 0x7fda9c0b1a80 2023-07-06T21:40:20.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.470+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda8c00cd60 con 0x7fda9c0b1a80 2023-07-06T21:40:20.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.471+0000 7fda8affd700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda7c06acb0 0x7fda7c06d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.471+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fda8c089610 con 0x7fda9c0b1a80 2023-07-06T21:40:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.471+0000 7fda99d9b700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda7c06acb0 0x7fda7c06d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.471+0000 7fda99d9b700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda7c06acb0 0x7fda7c06d160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda9c15c610 tx=0x7fda90008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.475+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda8c0545e0 con 0x7fda9c0b1a80 2023-07-06T21:40:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.670+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda9c04e1d0 con 0x7fda9c0b1a80 2023-07-06T21:40:20.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.671+0000 7fda8affd700 1 -- 172.21.15.84:0/493937794 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda8c057c00 con 0x7fda9c0b1a80 2023-07-06T21:40:20.676 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda7c06acb0 msgr2=0x7fda7c06d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda7c06acb0 0x7fda7c06d160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fda9c15c610 tx=0x7fda90008030 comp rx=0 tx=0).stop 2023-07-06T21:40:20.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 msgr2=0x7fda9c15aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 0x7fda9c15aa70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00e910 tx=0x7fda8c00ecd0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 shutdown_connections 2023-07-06T21:40:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda7c06acb0 0x7fda7c06d160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda9c0ceda0 0x7fda9c1e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda9c0b1a80 0x7fda9c15aa70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 --2- 172.21.15.84:0/493937794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda9c0b81c0 0x7fda9c0b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 >> 172.21.15.84:0/493937794 conn(0x7fda9c06b2b0 msgr2=0x7fda9c0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:20.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 shutdown_connections 2023-07-06T21:40:20.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.673+0000 7fdaa09ff700 1 -- 172.21.15.84:0/493937794 wait complete. 2023-07-06T21:40:20.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:20.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.884+0000 7f31ffbff700 1 Processor -- start 2023-07-06T21:40:20.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.885+0000 7f31ffbff700 1 -- start start 2023-07-06T21:40:20.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31ffbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f31f80d3710 0x7f31f80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31ffbff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f80cd910 con 0x7f31f80b83c0 2023-07-06T21:40:20.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31ffbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f80d0a80 con 0x7f31f80cfa90 2023-07-06T21:40:20.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31ffbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f80cff10 con 0x7f31f80d3710 2023-07-06T21:40:20.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31fd99b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f31f80b83c0 0x7f31f80cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36382/0 (socket says 172.21.15.84:36382) 2023-07-06T21:40:20.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31fd99b700 1 -- 172.21.15.84:0/1954648159 learned_addr learned my addr 172.21.15.84:0/1954648159 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:20.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794106818 0 0) 0x7f31f4001110 con 0x7f31f80b83c0 2023-07-06T21:40:20.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31e40035b0 con 0x7f31f80b83c0 2023-07-06T21:40:20.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.886+0000 7f31fd19a700 1 --2- 172.21.15.84:0/1954648159 >> v2:172.21.15.107:3300/0 conn(0x7f31f80d3710 0x7f31f80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397295851 0 0) 0x7f31e40035b0 con 0x7f31f80b83c0 2023-07-06T21:40:20.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 >> v2:172.21.15.107:3300/0 conn(0x7f31f80d3710 msgr2=0x7f31f80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:20.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 --2- 172.21.15.84:0/1954648159 >> v2:172.21.15.107:3300/0 conn(0x7f31f80d3710 0x7f31f80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 >> v1:172.21.15.107:6789/0 conn(0x7f31f80cfa90 legacy=0x7f31f80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f80b8790 con 0x7f31f80b83c0 2023-07-06T21:40:20.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f31f4001800 con 0x7f31f80b83c0 2023-07-06T21:40:20.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 >> v1:172.21.15.84:6789/0 conn(0x7f31f80b83c0 legacy=0x7f31f80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e4004850 0x7f31e4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31e4005200 0x7f31e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80cff10 con 0x7f31e4003a00 2023-07-06T21:40:20.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e400c110 con 0x7f31e4004850 2023-07-06T21:40:20.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e400c2d0 con 0x7f31e4005200 2023-07-06T21:40:20.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fd99b700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fd19a700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e4004850 0x7f31e4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fe19c700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31e4005200 0x7f31e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.887+0000 7f31fd99b700 1 -- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31e4005200 msgr2=0x7f31e400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:20.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fd99b700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31e4005200 0x7f31e400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fd99b700 1 -- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e4004850 msgr2=0x7f31e4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:20.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fd99b700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31e4004850 0x7f31e4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fd99b700 1 -- 172.21.15.84:0/1954648159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e40035b0 con 0x7f31e4003a00 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fe19c700 1 -- 172.21.15.84:0/1954648159 reap_dead start 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fd99b700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31e4003e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c240 tx=0x7f31e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff38186d7adf5a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4007db0 con 0x7f31e4003a00 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f400ce00 con 0x7f31e4003a00 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31fc999700 1 -- 172.21.15.84:0/1954648159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4005ba0 con 0x7f31e4003a00 2023-07-06T21:40:20.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31ffbff700 1 -- 172.21.15.84:0/1954648159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 msgr2=0x7f31e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.888+0000 7f31ffbff700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31e4003e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c240 tx=0x7f31e4005880 comp rx=0 tx=0).stop 2023-07-06T21:40:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.889+0000 7f31ffbff700 1 -- 172.21.15.84:0/1954648159 shutdown_connections 2023-07-06T21:40:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.889+0000 7f31ffbff700 1 --2- 172.21.15.84:0/1954648159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31e4003e10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.889+0000 7f31ffbff700 1 -- 172.21.15.84:0/1954648159 >> 172.21.15.84:0/1954648159 conn(0x7f31f806b350 msgr2=0x7f31f806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.889+0000 7f31ffbff700 1 -- 172.21.15.84:0/1954648159 shutdown_connections 2023-07-06T21:40:20.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.889+0000 7f31ffbff700 1 -- 172.21.15.84:0/1954648159 wait complete. 2023-07-06T21:40:20.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.890+0000 7f31ffbff700 1 Processor -- start 2023-07-06T21:40:20.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.890+0000 7f31ffbff700 1 -- start start 2023-07-06T21:40:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.891+0000 7f31ffbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31f80c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.891+0000 7f31ffbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31f815f480 0x7f31f8147e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.891+0000 7f31ffbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31f80d29d0 0x7f31f80d2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.891+0000 7f31ffbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80d33b0 con 0x7f31e4003a00 2023-07-06T21:40:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.891+0000 7f31ffbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80cccc0 con 0x7f31f80d29d0 2023-07-06T21:40:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.891+0000 7f31ffbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f815f8f0 con 0x7f31f815f480 2023-07-06T21:40:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31f80c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31f80c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43728/0 (socket says 172.21.15.84:43728) 2023-07-06T21:40:20.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 -- 172.21.15.84:0/961885617 learned_addr learned my addr 172.21.15.84:0/961885617 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:20.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd19a700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31f815f480 0x7f31f8147e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fe19c700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31f80d29d0 0x7f31f80d2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 -- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31f815f480 msgr2=0x7f31f8147e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31f815f480 0x7f31f8147e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 -- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31f80d29d0 msgr2=0x7f31f80d2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31f80d29d0 0x7f31f80d2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 -- 172.21.15.84:0/961885617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f80ca2b0 con 0x7f31e4003a00 2023-07-06T21:40:20.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fe19c700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31f80d29d0 0x7f31f80d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31fd99b700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31f80c9c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c830 tx=0x7f31f4001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.892+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f40017c0 con 0x7f31e4003a00 2023-07-06T21:40:20.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.893+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f40070d0 con 0x7f31e4003a00 2023-07-06T21:40:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.893+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f80cc420 con 0x7f31e4003a00 2023-07-06T21:40:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.893+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f4007480 con 0x7f31e4003a00 2023-07-06T21:40:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.893+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f80cc9e0 con 0x7f31e4003a00 2023-07-06T21:40:20.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.894+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f80b49e0 con 0x7f31e4003a00 2023-07-06T21:40:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.895+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f31f40075e0 con 0x7f31e4003a00 2023-07-06T21:40:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.896+0000 7f31ee7fc700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31dc06ad10 0x7f31dc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.896+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f31f408a6f0 con 0x7f31e4003a00 2023-07-06T21:40:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.896+0000 7f31fd19a700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31dc06ad10 0x7f31dc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:20.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.897+0000 7f31fd19a700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31dc06ad10 0x7f31dc06d1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f31f80cec70 tx=0x7f31e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:20.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:20.900+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f31f4055750 con 0x7f31e4003a00 2023-07-06T21:40:21.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.111+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f31f80cd910 con 0x7f31e4003a00 2023-07-06T21:40:21.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.111+0000 7f31ee7fc700 1 -- 172.21.15.84:0/961885617 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f31f80cd910 con 0x7f31e4003a00 2023-07-06T21:40:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.113+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31dc06ad10 msgr2=0x7f31dc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.113+0000 7f31ffbff700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31dc06ad10 0x7f31dc06d1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f31f80cec70 tx=0x7f31e8008040 comp rx=0 tx=0).stop 2023-07-06T21:40:21.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.113+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 msgr2=0x7f31f80c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.113+0000 7f31ffbff700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31f80c9c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f31f400c830 tx=0x7f31f4001b90 comp rx=0 tx=0).stop 2023-07-06T21:40:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 shutdown_connections 2023-07-06T21:40:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f31dc06ad10 0x7f31dc06d1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31f80d29d0 0x7f31f80d2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31e4003a00 0x7f31f80c9c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 --2- 172.21.15.84:0/961885617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31f815f480 0x7f31f8147e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 >> 172.21.15.84:0/961885617 conn(0x7f31f806b350 msgr2=0x7f31f80b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 shutdown_connections 2023-07-06T21:40:21.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.114+0000 7f31ffbff700 1 -- 172.21.15.84:0/961885617 wait complete. 2023-07-06T21:40:21.120 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:21.134 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:40:21.134 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:21.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:21.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.372+0000 7f0de19ff700 1 Processor -- start 2023-07-06T21:40:21.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.373+0000 7f0de19ff700 1 -- start start 2023-07-06T21:40:21.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0de19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0b83c0 0x7f0ddc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0de19ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ddc0d3710 0x7f0ddc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0de19ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0ddc0cfa90 0x7f0ddc15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0de19ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc0cfee0 con 0x7f0ddc0d3710 2023-07-06T21:40:21.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0de19ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ddc0d0a80 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0de19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc0b8790 con 0x7f0ddc0b83c0 2023-07-06T21:40:21.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ddc0d3710 0x7f0ddc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0ddc0d3710 0x7f0ddc0cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43734/0 (socket says 172.21.15.84:43734) 2023-07-06T21:40:21.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1045359729 learned_addr learned my addr 172.21.15.84:0/1045359729 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:21.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0ddb7fe700 1 --2- 172.21.15.84:0/1045359729 >> v2:172.21.15.107:3300/0 conn(0x7f0ddc0cfa90 0x7f0ddc15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0ddaffd700 1 --2- 172.21.15.84:0/1045359729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0b83c0 0x7f0ddc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1045359729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0b83c0 msgr2=0x7f0ddc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:21.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 --2- 172.21.15.84:0/1045359729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0b83c0 0x7f0ddc0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1045359729 >> v2:172.21.15.107:3300/0 conn(0x7f0ddc0cfa90 msgr2=0x7f0ddc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 --2- 172.21.15.84:0/1045359729 >> v2:172.21.15.107:3300/0 conn(0x7f0ddc0cfa90 0x7f0ddc15feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.374+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1045359729 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0b4e70 con 0x7f0ddc0d3710 2023-07-06T21:40:21.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0dda7fc700 1 --2- 172.21.15.84:0/1045359729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0d3710 0x7f0ddc0cf550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0ce5b0 tx=0x7f0dd0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfc91e83fad9d19 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:21.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0dd9ffb700 1 -- 172.21.15.84:0/1045359729 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000b870 con 0x7f0ddc0d3710 2023-07-06T21:40:21.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0dd9ffb700 1 -- 172.21.15.84:0/1045359729 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd000b9d0 con 0x7f0ddc0d3710 2023-07-06T21:40:21.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0dd9ffb700 1 -- 172.21.15.84:0/1045359729 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd000bd60 con 0x7f0ddc0d3710 2023-07-06T21:40:21.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 -- 172.21.15.84:0/1045359729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0d3710 msgr2=0x7f0ddc0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1045359729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0d3710 0x7f0ddc0cf550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0ce5b0 tx=0x7f0dd0009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 -- 172.21.15.84:0/1045359729 shutdown_connections 2023-07-06T21:40:21.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1045359729 >> v2:172.21.15.107:3300/0 conn(0x7f0ddc0cfa90 0x7f0ddc15feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1045359729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0b83c0 0x7f0ddc0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1045359729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0d3710 0x7f0ddc0cf550 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.375+0000 7f0de19ff700 1 -- 172.21.15.84:0/1045359729 >> 172.21.15.84:0/1045359729 conn(0x7f0ddc06b350 msgr2=0x7f0ddc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:21.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.376+0000 7f0de19ff700 1 -- 172.21.15.84:0/1045359729 shutdown_connections 2023-07-06T21:40:21.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.376+0000 7f0de19ff700 1 -- 172.21.15.84:0/1045359729 wait complete. 2023-07-06T21:40:21.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.377+0000 7f0de19ff700 1 Processor -- start 2023-07-06T21:40:21.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.377+0000 7f0de19ff700 1 -- start start 2023-07-06T21:40:21.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.378+0000 7f0de19ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ddc0b83c0 0x7f0ddc148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.378+0000 7f0de19ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 0x7f0ddc15d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.378+0000 7f0de19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0d3710 0x7f0ddc14b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.378+0000 7f0de19ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc15e2c0 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.378+0000 7f0de19ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc15e5a0 con 0x7f0ddc0b83c0 2023-07-06T21:40:21.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.378+0000 7f0de19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ddc15e880 con 0x7f0ddc0d3710 2023-07-06T21:40:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 0x7f0ddc15d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 0x7f0ddc15d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43736/0 (socket says 172.21.15.84:43736) 2023-07-06T21:40:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1556196487 learned_addr learned my addr 172.21.15.84:0/1556196487 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0ddaffd700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ddc0b83c0 0x7f0ddc148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0ddb7fe700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0d3710 0x7f0ddc14b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0d3710 msgr2=0x7f0ddc14b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0d3710 0x7f0ddc14b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ddc0b83c0 msgr2=0x7f0ddc148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ddc0b83c0 0x7f0ddc148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 -- 172.21.15.84:0/1556196487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0009550 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0ddb7fe700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0d3710 0x7f0ddc14b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dda7fc700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 0x7f0ddc15d940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:21.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.379+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd0004520 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.380+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd001e070 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.380+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ddc15eb60 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.380+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd00038c0 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.380+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ddc149250 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.381+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ddc0d1470 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.382+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0dd0003a20 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.382+0000 7f0dc3fff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0dc806ace0 0x7f0dc806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.383+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0dd0020070 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.383+0000 7f0ddaffd700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0dc806ace0 0x7f0dc806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.383+0000 7f0ddaffd700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0dc806ace0 0x7f0dc806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc006b20 tx=0x7f0dcc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:21.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.387+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0dd0055a50 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.583+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ddc0d1190 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.584+0000 7f0dc3fff700 1 -- 172.21.15.84:0/1556196487 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f0dd0059070 con 0x7f0ddc0cfa90 2023-07-06T21:40:21.587 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:21.587 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:21.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0dc806ace0 msgr2=0x7f0dc806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0dc806ace0 0x7f0dc806d190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc006b20 tx=0x7f0dcc008040 comp rx=0 tx=0).stop 2023-07-06T21:40:21.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 msgr2=0x7f0ddc15d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 0x7f0ddc15d940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0000c00 tx=0x7f0dd0004750 comp rx=0 tx=0).stop 2023-07-06T21:40:21.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 shutdown_connections 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0dc806ace0 0x7f0dc806d190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ddc0d3710 0x7f0ddc14b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ddc0b83c0 0x7f0ddc148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 --2- 172.21.15.84:0/1556196487 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0ddc0cfa90 0x7f0ddc15d940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 >> 172.21.15.84:0/1556196487 conn(0x7f0ddc06b350 msgr2=0x7f0ddc0d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 shutdown_connections 2023-07-06T21:40:21.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.587+0000 7f0de19ff700 1 -- 172.21.15.84:0/1556196487 wait complete. 2023-07-06T21:40:21.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:21.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:21.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.803+0000 7f0a701ff700 1 Processor -- start 2023-07-06T21:40:21.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.803+0000 7f0a701ff700 1 -- start start 2023-07-06T21:40:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.804+0000 7f0a701ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6806ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.804+0000 7f0a701ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a6806dde0 0x7f0a6806a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.804+0000 7f0a701ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a680c94b0 con 0x7f0a680cf280 2023-07-06T21:40:21.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.804+0000 7f0a701ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a6806d350 con 0x7f0a6806dde0 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.804+0000 7f0a701ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6806a7a0 con 0x7f0a68069e90 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6d79a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0a6806dde0 0x7f0a6806a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6e79c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0a680cf280 0x7f0a680b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36386/0 (socket says 172.21.15.84:36386) 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6e79c700 1 -- 172.21.15.84:0/4264986003 learned_addr learned my addr 172.21.15.84:0/4264986003 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6df9b700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6806ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6cf99700 1 -- 172.21.15.84:0/4264986003 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334412019 0 0) 0x7f0a680c94b0 con 0x7f0a680cf280 2023-07-06T21:40:21.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6cf99700 1 -- 172.21.15.84:0/4264986003 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a540035b0 con 0x7f0a680cf280 2023-07-06T21:40:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6d79a700 1 -- 172.21.15.84:0/4264986003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 msgr2=0x7f0a6806ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6d79a700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6806ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6d79a700 1 -- 172.21.15.84:0/4264986003 >> v1:172.21.15.84:6789/0 conn(0x7f0a680cf280 legacy=0x7f0a680b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6d79a700 1 -- 172.21.15.84:0/4264986003 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680ca9b0 con 0x7f0a6806dde0 2023-07-06T21:40:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.805+0000 7f0a6df9b700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a6d79a700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6806a260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009840 tx=0x7f0a58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5593e3c56ddf6e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a6cf99700 1 -- 172.21.15.84:0/4264986003 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58011830 con 0x7f0a6806dde0 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a6cf99700 1 -- 172.21.15.84:0/4264986003 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a58011990 con 0x7f0a6806dde0 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a6cf99700 1 -- 172.21.15.84:0/4264986003 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58011d20 con 0x7f0a6806dde0 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 -- 172.21.15.84:0/4264986003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 msgr2=0x7f0a6806a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6806a260 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009840 tx=0x7f0a58009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:21.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 -- 172.21.15.84:0/4264986003 shutdown_connections 2023-07-06T21:40:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 --2- 172.21.15.84:0/4264986003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6806a260 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 -- 172.21.15.84:0/4264986003 >> 172.21.15.84:0/4264986003 conn(0x7f0a680cbf10 msgr2=0x7f0a680ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.806+0000 7f0a701ff700 1 -- 172.21.15.84:0/4264986003 shutdown_connections 2023-07-06T21:40:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.807+0000 7f0a701ff700 1 -- 172.21.15.84:0/4264986003 wait complete. 2023-07-06T21:40:21.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.808+0000 7f0a701ff700 1 Processor -- start 2023-07-06T21:40:21.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.808+0000 7f0a701ff700 1 -- start start 2023-07-06T21:40:21.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a701ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6814e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a701ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6815ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a701ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a680cf280 0x7f0a680c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a701ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6815b8d0 con 0x7f0a6806dde0 2023-07-06T21:40:21.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a701ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6815bbb0 con 0x7f0a680cf280 2023-07-06T21:40:21.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a701ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680af570 con 0x7f0a68069e90 2023-07-06T21:40:21.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a6d79a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6815ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a6d79a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6815ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43754/0 (socket says 172.21.15.84:43754) 2023-07-06T21:40:21.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.809+0000 7f0a6d79a700 1 -- 172.21.15.84:0/338747882 learned_addr learned my addr 172.21.15.84:0/338747882 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:21.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6df9b700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6814e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6e79c700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a680cf280 0x7f0a680c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6d79a700 1 -- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 msgr2=0x7f0a6814e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6d79a700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6814e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6d79a700 1 -- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a680cf280 msgr2=0x7f0a680c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:21.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6d79a700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a680cf280 0x7f0a680c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:21.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6d79a700 1 -- 172.21.15.84:0/338747882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a580094a0 con 0x7f0a6806dde0 2023-07-06T21:40:21.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6e79c700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a680cf280 0x7f0a680c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:21.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a6d79a700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6815ae80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0a580090b0 tx=0x7f0a58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:21.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a580036a0 con 0x7f0a6806dde0 2023-07-06T21:40:21.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5801e070 con 0x7f0a6806dde0 2023-07-06T21:40:21.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a680af850 con 0x7f0a6806dde0 2023-07-06T21:40:21.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.810+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a58003a60 con 0x7f0a6806dde0 2023-07-06T21:40:21.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.811+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a6815d680 con 0x7f0a6806dde0 2023-07-06T21:40:21.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.812+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680afa60 con 0x7f0a6806dde0 2023-07-06T21:40:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.813+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0a58003bc0 con 0x7f0a6806dde0 2023-07-06T21:40:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.813+0000 7f0a5e7fc700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a4c06ace0 0x7f0a4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.814+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f0a5808a370 con 0x7f0a6806dde0 2023-07-06T21:40:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.814+0000 7f0a6df9b700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a4c06ace0 0x7f0a4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:21.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.814+0000 7f0a6df9b700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a4c06ace0 0x7f0a4c06d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c940 tx=0x7f0a64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:21.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:21.818+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0a58055470 con 0x7f0a6806dde0 2023-07-06T21:40:22.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.016+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a6804e1d0 con 0x7f0a6806dde0 2023-07-06T21:40:22.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.016+0000 7f0a5e7fc700 1 -- 172.21.15.84:0/338747882 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a58058a90 con 0x7f0a6806dde0 2023-07-06T21:40:22.157 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:22.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a4c06ace0 msgr2=0x7f0a4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a4c06ace0 0x7f0a4c06d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a6400c940 tx=0x7f0a64008040 comp rx=0 tx=0).stop 2023-07-06T21:40:22.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 msgr2=0x7f0a6815ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6815ae80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0a580090b0 tx=0x7f0a58004750 comp rx=0 tx=0).stop 2023-07-06T21:40:22.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 shutdown_connections 2023-07-06T21:40:22.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0a4c06ace0 0x7f0a4c06d190 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a680cf280 0x7f0a680c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a68069e90 0x7f0a6814e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 --2- 172.21.15.84:0/338747882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0a6806dde0 0x7f0a6815ae80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 >> 172.21.15.84:0/338747882 conn(0x7f0a680cbf10 msgr2=0x7f0a680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:22.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 shutdown_connections 2023-07-06T21:40:22.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.018+0000 7f0a701ff700 1 -- 172.21.15.84:0/338747882 wait complete. 2023-07-06T21:40:22.162 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:22.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.358+0000 7ff91c3ff700 1 Processor -- start 2023-07-06T21:40:22.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.358+0000 7ff91c3ff700 1 -- start start 2023-07-06T21:40:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.359+0000 7ff91c3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff91415c420 0x7ff914070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.359+0000 7ff91c3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91406bf00 con 0x7ff9140ce730 2023-07-06T21:40:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.359+0000 7ff91c3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9140cb3b0 con 0x7ff9140caf40 2023-07-06T21:40:22.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.359+0000 7ff91c3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91406b9b0 con 0x7ff91415c420 2023-07-06T21:40:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a99c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff9140ce730 0x7ff9140b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36398/0 (socket says 172.21.15.84:36398) 2023-07-06T21:40:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a99c700 1 -- 172.21.15.84:0/2187031892 learned_addr learned my addr 172.21.15.84:0/2187031892 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:22.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a19b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff91415c420 0x7ff914070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a19b700 1 -- 172.21.15.84:0/2187031892 >> v1:172.21.15.107:6789/0 conn(0x7ff9140caf40 legacy=0x7ff91406c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a19b700 1 -- 172.21.15.84:0/2187031892 >> v1:172.21.15.84:6789/0 conn(0x7ff9140ce730 legacy=0x7ff9140b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff919199700 1 -- 172.21.15.84:0/2187031892 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777798240 0 0) 0x7ff91406bf00 con 0x7ff9140ce730 2023-07-06T21:40:22.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a19b700 1 -- 172.21.15.84:0/2187031892 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140cebb0 con 0x7ff91415c420 2023-07-06T21:40:22.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff91a19b700 1 --2- 172.21.15.84:0/2187031892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91415c420 0x7ff914070af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff910009840 tx=0x7ff910009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc23efa8c611418 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:22.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.360+0000 7ff919199700 1 -- 172.21.15.84:0/2187031892 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910011830 con 0x7ff91415c420 2023-07-06T21:40:22.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff919199700 1 -- 172.21.15.84:0/2187031892 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910011990 con 0x7ff91415c420 2023-07-06T21:40:22.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff919199700 1 -- 172.21.15.84:0/2187031892 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910011d20 con 0x7ff91415c420 2023-07-06T21:40:22.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 -- 172.21.15.84:0/2187031892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91415c420 msgr2=0x7ff914070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/2187031892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91415c420 0x7ff914070af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff910009840 tx=0x7ff910009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:22.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 -- 172.21.15.84:0/2187031892 shutdown_connections 2023-07-06T21:40:22.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/2187031892 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff91415c420 0x7ff914070af0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 -- 172.21.15.84:0/2187031892 >> 172.21.15.84:0/2187031892 conn(0x7ff91406e540 msgr2=0x7ff91406e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:22.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 -- 172.21.15.84:0/2187031892 shutdown_connections 2023-07-06T21:40:22.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.361+0000 7ff91c3ff700 1 -- 172.21.15.84:0/2187031892 wait complete. 2023-07-06T21:40:22.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.363+0000 7ff91c3ff700 1 Processor -- start 2023-07-06T21:40:22.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.363+0000 7ff91c3ff700 1 -- start start 2023-07-06T21:40:22.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91c3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 0x7ff914157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9140ce730 0x7ff91415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91c3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff91415c420 0x7ff9141573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91c3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91415f890 con 0x7ff9140caf40 2023-07-06T21:40:22.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91c3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91415f5c0 con 0x7ff91415c420 2023-07-06T21:40:22.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91415d730 con 0x7ff9140ce730 2023-07-06T21:40:22.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 0x7ff914157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 0x7ff914157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43778/0 (socket says 172.21.15.84:43778) 2023-07-06T21:40:22.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 -- 172.21.15.84:0/1824115338 learned_addr learned my addr 172.21.15.84:0/1824115338 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:22.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 -- 172.21.15.84:0/1824115338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9140ce730 msgr2=0x7ff91415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9140ce730 0x7ff91415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 -- 172.21.15.84:0/1824115338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff91415c420 msgr2=0x7ff9141573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff91415c420 0x7ff9141573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.364+0000 7ff91a19b700 1 -- 172.21.15.84:0/1824115338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100094a0 con 0x7ff9140caf40 2023-07-06T21:40:22.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.365+0000 7ff91a19b700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 0x7ff914157f90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff9100090b0 tx=0x7ff910004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:22.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.365+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91001a980 con 0x7ff9140caf40 2023-07-06T21:40:22.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.365+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91001ab00 con 0x7ff9140caf40 2023-07-06T21:40:22.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.365+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91415da10 con 0x7ff9140caf40 2023-07-06T21:40:22.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.365+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910024420 con 0x7ff9140caf40 2023-07-06T21:40:22.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.365+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91415dfd0 con 0x7ff9140caf40 2023-07-06T21:40:22.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.366+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91404e1d0 con 0x7ff9140caf40 2023-07-06T21:40:22.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.367+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff910022070 con 0x7ff9140caf40 2023-07-06T21:40:22.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.368+0000 7ff90affd700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff8f806ace0 0x7ff8f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.369+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff91008a900 con 0x7ff9140caf40 2023-07-06T21:40:22.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.369+0000 7ff91999a700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff8f806ace0 0x7ff8f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.369+0000 7ff91999a700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff8f806ace0 0x7ff8f806d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff904000f60 tx=0x7ff904007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:22.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.373+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff910055960 con 0x7ff9140caf40 2023-07-06T21:40:22.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.574+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff91406f420 con 0x7ff9140caf40 2023-07-06T21:40:22.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.575+0000 7ff90affd700 1 -- 172.21.15.84:0/1824115338 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff9100290a0 con 0x7ff9140caf40 2023-07-06T21:40:22.578 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff8f806ace0 msgr2=0x7ff8f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff8f806ace0 0x7ff8f806d190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff904000f60 tx=0x7ff904007a40 comp rx=0 tx=0).stop 2023-07-06T21:40:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 msgr2=0x7ff914157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 0x7ff914157f90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff9100090b0 tx=0x7ff910004750 comp rx=0 tx=0).stop 2023-07-06T21:40:22.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 shutdown_connections 2023-07-06T21:40:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff8f806ace0 0x7ff8f806d190 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff91415c420 0x7ff9141573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9140caf40 0x7ff914157f90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 --2- 172.21.15.84:0/1824115338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9140ce730 0x7ff91415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.577+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 >> 172.21.15.84:0/1824115338 conn(0x7ff91406e540 msgr2=0x7ff9140d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:22.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.578+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 shutdown_connections 2023-07-06T21:40:22.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.578+0000 7ff91c3ff700 1 -- 172.21.15.84:0/1824115338 wait complete. 2023-07-06T21:40:22.598 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:22.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.793+0000 7f544c1ff700 1 Processor -- start 2023-07-06T21:40:22.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.794+0000 7f544c1ff700 1 -- start start 2023-07-06T21:40:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544c1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54440b81c0 0x7f54440cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544c1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f54440b1a80 0x7f54440cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544c1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54440d1830 con 0x7f54440ceda0 2023-07-06T21:40:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544c1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440cf330 con 0x7f54440b1a80 2023-07-06T21:40:22.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544c1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440d1270 con 0x7f54440b81c0 2023-07-06T21:40:22.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544a79c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f54440ceda0 0x7f544415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:36414/0 (socket says 172.21.15.84:36414) 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544a79c700 1 -- 172.21.15.84:0/1259096144 learned_addr learned my addr 172.21.15.84:0/1259096144 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544979a700 1 --2- 172.21.15.84:0/1259096144 >> v2:172.21.15.84:3300/0 conn(0x7f54440b1a80 0x7f54440cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f5449f9b700 1 --2- 172.21.15.84:0/1259096144 >> v2:172.21.15.107:3300/0 conn(0x7f54440b81c0 0x7f54440cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544979a700 1 -- 172.21.15.84:0/1259096144 >> v2:172.21.15.107:3300/0 conn(0x7f54440b81c0 msgr2=0x7f54440cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544979a700 1 --2- 172.21.15.84:0/1259096144 >> v2:172.21.15.107:3300/0 conn(0x7f54440b81c0 0x7f54440cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.795+0000 7f544979a700 1 -- 172.21.15.84:0/1259096144 >> v1:172.21.15.84:6789/0 conn(0x7f54440ceda0 legacy=0x7f544415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f544979a700 1 -- 172.21.15.84:0/1259096144 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440d1550 con 0x7f54440b1a80 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f544979a700 1 --2- 172.21.15.84:0/1259096144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440b1a80 0x7f54440cfb40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54440b2620 tx=0x7f5434009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8fe68942936f76 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f5448f99700 1 -- 172.21.15.84:0/1259096144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434011830 con 0x7f54440b1a80 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f5448f99700 1 -- 172.21.15.84:0/1259096144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434011990 con 0x7f54440b1a80 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f5448f99700 1 -- 172.21.15.84:0/1259096144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434011d20 con 0x7f54440b1a80 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f544c1ff700 1 -- 172.21.15.84:0/1259096144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440b1a80 msgr2=0x7f54440cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.796+0000 7f544c1ff700 1 --2- 172.21.15.84:0/1259096144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440b1a80 0x7f54440cfb40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54440b2620 tx=0x7f5434009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:22.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.797+0000 7f544c1ff700 1 -- 172.21.15.84:0/1259096144 shutdown_connections 2023-07-06T21:40:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.797+0000 7f544c1ff700 1 --2- 172.21.15.84:0/1259096144 >> v2:172.21.15.107:3300/0 conn(0x7f54440b81c0 0x7f54440cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.797+0000 7f544c1ff700 1 --2- 172.21.15.84:0/1259096144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440b1a80 0x7f54440cfb40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.797+0000 7f544c1ff700 1 -- 172.21.15.84:0/1259096144 >> 172.21.15.84:0/1259096144 conn(0x7f544406b2b0 msgr2=0x7f544406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.797+0000 7f544c1ff700 1 -- 172.21.15.84:0/1259096144 shutdown_connections 2023-07-06T21:40:22.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.797+0000 7f544c1ff700 1 -- 172.21.15.84:0/1259096144 wait complete. 2023-07-06T21:40:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.798+0000 7f544c1ff700 1 Processor -- start 2023-07-06T21:40:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 -- start start 2023-07-06T21:40:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54440b1a80 0x7f54440caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54440b81c0 0x7f54440d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 0x7f54440c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440ca4b0 con 0x7f54440ceda0 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440caa70 con 0x7f54440b1a80 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.799+0000 7f544c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54440c71e0 con 0x7f54440b81c0 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 0x7f54440c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 0x7f54440c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43804/0 (socket says 172.21.15.84:43804) 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 -- 172.21.15.84:0/60155354 learned_addr learned my addr 172.21.15.84:0/60155354 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544979a700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54440b81c0 0x7f54440d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f5449f9b700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54440b1a80 0x7f54440caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 -- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54440b81c0 msgr2=0x7f54440d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54440b81c0 0x7f54440d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 -- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54440b1a80 msgr2=0x7f54440caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54440b1a80 0x7f54440caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 -- 172.21.15.84:0/60155354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54340094a0 con 0x7f54440ceda0 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544979a700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54440b81c0 0x7f54440d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.800+0000 7f544a79c700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 0x7f54440c6720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f543c00cbf0 tx=0x7f543c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:22.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.801+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00ab10 con 0x7f54440ceda0 2023-07-06T21:40:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.801+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c00ac70 con 0x7f54440ceda0 2023-07-06T21:40:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.801+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54440c74c0 con 0x7f54440ceda0 2023-07-06T21:40:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.801+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c012400 con 0x7f54440ceda0 2023-07-06T21:40:22.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.801+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54440ccc40 con 0x7f54440ceda0 2023-07-06T21:40:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.802+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54440cb440 con 0x7f54440ceda0 2023-07-06T21:40:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.803+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f543c01b3b0 con 0x7f54440ceda0 2023-07-06T21:40:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.804+0000 7f543a7fc700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f542806ace0 0x7f542806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.804+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f543c0890d0 con 0x7f54440ceda0 2023-07-06T21:40:22.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.804+0000 7f5449f9b700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f542806ace0 0x7f542806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:22.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.805+0000 7f5449f9b700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f542806ace0 0x7f542806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5440006b20 tx=0x7f5440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:22.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:22.809+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f543c054150 con 0x7f54440ceda0 2023-07-06T21:40:23.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.005+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54440d3360 con 0x7f54440ceda0 2023-07-06T21:40:23.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.005+0000 7f543a7fc700 1 -- 172.21.15.84:0/60155354 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f543c057770 con 0x7f54440ceda0 2023-07-06T21:40:23.008 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:23.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f542806ace0 msgr2=0x7f542806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f542806ace0 0x7f542806d190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5440006b20 tx=0x7f5440008040 comp rx=0 tx=0).stop 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 msgr2=0x7f54440c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 0x7f54440c6720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f543c00cbf0 tx=0x7f543c00cf00 comp rx=0 tx=0).stop 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 shutdown_connections 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f542806ace0 0x7f542806d190 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54440ceda0 0x7f54440c6720 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54440b1a80 0x7f54440caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 --2- 172.21.15.84:0/60155354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54440b81c0 0x7f54440d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 >> 172.21.15.84:0/60155354 conn(0x7f544406b2b0 msgr2=0x7f54440b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:23.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 shutdown_connections 2023-07-06T21:40:23.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.007+0000 7f544c1ff700 1 -- 172.21.15.84:0/60155354 wait complete. 2023-07-06T21:40:23.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:23.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.225+0000 7fa006bff700 1 Processor -- start 2023-07-06T21:40:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.225+0000 7fa006bff700 1 -- start start 2023-07-06T21:40:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa006bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0000b81c0 0x7fa0000cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa006bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0000d1830 con 0x7fa0000ceda0 2023-07-06T21:40:23.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa006bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa000160d40 con 0x7fa0000b1a80 2023-07-06T21:40:23.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa006bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0000cf330 con 0x7fa0000b81c0 2023-07-06T21:40:23.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa00519c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa0000ceda0 0x7fa000160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46158/0 (socket says 172.21.15.84:46158) 2023-07-06T21:40:23.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa00519c700 1 -- 172.21.15.84:0/3831234639 learned_addr learned my addr 172.21.15.84:0/3831234639 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:23.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7fa00499b700 1 --2- 172.21.15.84:0/3831234639 >> v2:172.21.15.107:3300/0 conn(0x7fa0000b81c0 0x7fa0000cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1596286376 0 0) 0x7fa0000d1830 con 0x7fa0000ceda0 2023-07-06T21:40:23.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.227+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ff00035b0 con 0x7fa0000ceda0 2023-07-06T21:40:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107534863 0 0) 0x7f9ff00035b0 con 0x7fa0000ceda0 2023-07-06T21:40:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 >> v2:172.21.15.107:3300/0 conn(0x7fa0000b81c0 msgr2=0x7fa0000cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 --2- 172.21.15.84:0/3831234639 >> v2:172.21.15.107:3300/0 conn(0x7fa0000b81c0 0x7fa0000cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 >> v1:172.21.15.107:6789/0 conn(0x7fa0000b1a80 legacy=0x7fa00015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0000d1270 con 0x7fa0000ceda0 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f9ff80019f0 con 0x7fa0000ceda0 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 >> v1:172.21.15.84:6789/0 conn(0x7fa0000ceda0 legacy=0x7fa000160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0003a00 0x7f9ff0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ff0004850 0x7f9ff0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ff0005200 0x7f9ff000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0000cf330 con 0x7f9ff0005200 2023-07-06T21:40:23.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff000c080 con 0x7f9ff0004850 2023-07-06T21:40:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff000c240 con 0x7f9ff0003a00 2023-07-06T21:40:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7fa00519c700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ff0005200 0x7f9ff000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7fa00499b700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0003a00 0x7f9ff0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.228+0000 7f9ffffff700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ff0004850 0x7f9ff0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 -- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0003a00 msgr2=0x7f9ff0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0003a00 0x7f9ff0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 -- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ff0004850 msgr2=0x7f9ff0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:23.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ff0004850 0x7f9ff0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 -- 172.21.15.84:0/3831234639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00035b0 con 0x7f9ff0005200 2023-07-06T21:40:23.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7f9ffffff700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ff0004850 0x7f9ff0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:23.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 -- 172.21.15.84:0/3831234639 reap_dead start 2023-07-06T21:40:23.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7fa00519c700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ff0005200 0x7f9ff000ba10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800a830 tx=0x7f9ff0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c95068fa5700502 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:23.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff800a2f0 con 0x7f9ff0005200 2023-07-06T21:40:23.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff8012620 con 0x7f9ff0005200 2023-07-06T21:40:23.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.229+0000 7f9fff7fe700 1 -- 172.21.15.84:0/3831234639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff80129d0 con 0x7f9ff0005200 2023-07-06T21:40:23.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 -- 172.21.15.84:0/3831234639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ff0005200 msgr2=0x7f9ff000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ff0005200 0x7f9ff000ba10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800a830 tx=0x7f9ff0005880 comp rx=0 tx=0).stop 2023-07-06T21:40:23.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 -- 172.21.15.84:0/3831234639 shutdown_connections 2023-07-06T21:40:23.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 --2- 172.21.15.84:0/3831234639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ff0005200 0x7f9ff000ba10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 -- 172.21.15.84:0/3831234639 >> 172.21.15.84:0/3831234639 conn(0x7fa00006b2b0 msgr2=0x7fa00006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 -- 172.21.15.84:0/3831234639 shutdown_connections 2023-07-06T21:40:23.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.230+0000 7fa006bff700 1 -- 172.21.15.84:0/3831234639 wait complete. 2023-07-06T21:40:23.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.232+0000 7fa006bff700 1 Processor -- start 2023-07-06T21:40:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.232+0000 7fa006bff700 1 -- start start 2023-07-06T21:40:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa006bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0005200 0x7fa0000c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa006bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 0x7fa0001e8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa006bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0001e8560 0x7fa000158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa006bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0001e8b10 con 0x7fa000150540 2023-07-06T21:40:23.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa006bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015cbc0 con 0x7fa0001e8560 2023-07-06T21:40:23.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa006bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00015cea0 con 0x7f9ff0005200 2023-07-06T21:40:23.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 0x7fa0001e8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7f9ffffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 0x7fa0001e8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45892/0 (socket says 172.21.15.84:45892) 2023-07-06T21:40:23.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7f9ffffff700 1 -- 172.21.15.84:0/2733177358 learned_addr learned my addr 172.21.15.84:0/2733177358 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:23.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa00519c700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0001e8560 0x7fa000158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.233+0000 7fa00499b700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0005200 0x7fa0000c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffffff700 1 -- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0005200 msgr2=0x7fa0000c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffffff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0005200 0x7fa0000c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffffff700 1 -- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0001e8560 msgr2=0x7fa000158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffffff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0001e8560 0x7fa000158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffffff700 1 -- 172.21.15.84:0/2733177358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00015d180 con 0x7fa000150540 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffffff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 0x7fa0001e8020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00f860 tx=0x7f9fec00fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0052e0 con 0x7fa000150540 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec005440 con 0x7fa000150540 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0038c0 con 0x7fa000150540 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa00015d460 con 0x7fa000150540 2023-07-06T21:40:23.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.234+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0001513f0 con 0x7fa000150540 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.235+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00015d620 con 0x7fa000150540 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.236+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9fec003a20 con 0x7fa000150540 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.237+0000 7f9ffd7fa700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fe406ace0 0x7f9fe406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.237+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9fec0892a0 con 0x7fa000150540 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.238+0000 7fa00499b700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fe406ace0 0x7f9fe406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.238+0000 7fa00499b700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fe406ace0 0x7f9fe406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0001517d0 tx=0x7f9ff4009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:23.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.241+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9fec054310 con 0x7fa000150540 2023-07-06T21:40:23.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.439+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa00004e1d0 con 0x7fa000150540 2023-07-06T21:40:23.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.439+0000 7f9ffd7fa700 1 -- 172.21.15.84:0/2733177358 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fec057930 con 0x7fa000150540 2023-07-06T21:40:23.442 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.441+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fe406ace0 msgr2=0x7f9fe406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.441+0000 7fa006bff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fe406ace0 0x7f9fe406d190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa0001517d0 tx=0x7f9ff4009210 comp rx=0 tx=0).stop 2023-07-06T21:40:23.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.441+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 msgr2=0x7fa0001e8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.441+0000 7fa006bff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 0x7fa0001e8020 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00f860 tx=0x7f9fec00fb70 comp rx=0 tx=0).stop 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 shutdown_connections 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fe406ace0 0x7f9fe406d190 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0001e8560 0x7fa000158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ff0005200 0x7fa0000c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 --2- 172.21.15.84:0/2733177358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa000150540 0x7fa0001e8020 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 >> 172.21.15.84:0/2733177358 conn(0x7fa00006b2b0 msgr2=0x7fa000073380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:23.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 shutdown_connections 2023-07-06T21:40:23.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.442+0000 7fa006bff700 1 -- 172.21.15.84:0/2733177358 wait complete. 2023-07-06T21:40:23.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:23.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.654+0000 7f613f3ff700 1 Processor -- start 2023-07-06T21:40:23.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.654+0000 7f613f3ff700 1 -- start start 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.655+0000 7f613f3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6138070230 0x7f61380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.655+0000 7f613f3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61380c9c10 0x7f61380ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.655+0000 7f613f3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f613815d930 con 0x7f6138159130 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.655+0000 7f613f3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6138070bf0 con 0x7f6138070230 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.655+0000 7f613f3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f613815cd20 con 0x7f61380c9c10 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613d19b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6138159130 0x7f61380c4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46162/0 (socket says 172.21.15.84:46162) 2023-07-06T21:40:23.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613d19b700 1 -- 172.21.15.84:0/4121082440 learned_addr learned my addr 172.21.15.84:0/4121082440 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:23.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613d99c700 1 --2- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61380c9c10 0x7f61380ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613c99a700 1 --2- 172.21.15.84:0/4121082440 >> v2:172.21.15.107:3300/0 conn(0x7f6138070230 0x7f61380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f612ffff700 1 -- 172.21.15.84:0/4121082440 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101067780 0 0) 0x7f613815d930 con 0x7f6138159130 2023-07-06T21:40:23.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f612ffff700 1 -- 172.21.15.84:0/4121082440 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61240035b0 con 0x7f6138159130 2023-07-06T21:40:23.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613c99a700 1 -- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61380c9c10 msgr2=0x7f61380ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:23.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613c99a700 1 --2- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61380c9c10 0x7f61380ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613c99a700 1 -- 172.21.15.84:0/4121082440 >> v1:172.21.15.84:6789/0 conn(0x7f6138159130 legacy=0x7f61380c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.656+0000 7f613c99a700 1 -- 172.21.15.84:0/4121082440 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f613815d000 con 0x7f6138070230 2023-07-06T21:40:23.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f613c99a700 1 --2- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6138070230 0x7f61380706b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6128009840 tx=0x7f6128009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f1e8faae2943dc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:23.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f612ffff700 1 -- 172.21.15.84:0/4121082440 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128011830 con 0x7f6138070230 2023-07-06T21:40:23.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f612ffff700 1 -- 172.21.15.84:0/4121082440 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128011990 con 0x7f6138070230 2023-07-06T21:40:23.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f612ffff700 1 -- 172.21.15.84:0/4121082440 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128011d20 con 0x7f6138070230 2023-07-06T21:40:23.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f613f3ff700 1 -- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6138070230 msgr2=0x7f61380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f613f3ff700 1 --2- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6138070230 0x7f61380706b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6128009840 tx=0x7f6128009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:23.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f613f3ff700 1 -- 172.21.15.84:0/4121082440 shutdown_connections 2023-07-06T21:40:23.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f613f3ff700 1 --2- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61380c9c10 0x7f61380ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.657+0000 7f613f3ff700 1 --2- 172.21.15.84:0/4121082440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6138070230 0x7f61380706b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.658+0000 7f613f3ff700 1 -- 172.21.15.84:0/4121082440 >> 172.21.15.84:0/4121082440 conn(0x7f61380cf3d0 msgr2=0x7f61380cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:23.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.658+0000 7f613f3ff700 1 -- 172.21.15.84:0/4121082440 shutdown_connections 2023-07-06T21:40:23.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.658+0000 7f613f3ff700 1 -- 172.21.15.84:0/4121082440 wait complete. 2023-07-06T21:40:23.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.660+0000 7f613f3ff700 1 Processor -- start 2023-07-06T21:40:23.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.660+0000 7f613f3ff700 1 -- start start 2023-07-06T21:40:23.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.661+0000 7f613f3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6138070230 0x7f613806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.661+0000 7f613f3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61380c9c10 0x7f61380aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.661+0000 7f613f3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 0x7f61381f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.661+0000 7f613f3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380ae1e0 con 0x7f6138159130 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.661+0000 7f613f3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61381f2ce0 con 0x7f61380c9c10 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.661+0000 7f613f3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61381f2fc0 con 0x7f6138070230 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 0x7f61381f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 0x7f61381f21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45896/0 (socket says 172.21.15.84:45896) 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 -- 172.21.15.84:0/3391427366 learned_addr learned my addr 172.21.15.84:0/3391427366 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d19b700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6138070230 0x7f613806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613c99a700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61380c9c10 0x7f61380aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 -- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6138070230 msgr2=0x7f613806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6138070230 0x7f613806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 -- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61380c9c10 msgr2=0x7f61380aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61380c9c10 0x7f61380aebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 -- 172.21.15.84:0/3391427366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61280094a0 con 0x7f6138159130 2023-07-06T21:40:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d19b700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6138070230 0x7f613806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613c99a700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61380c9c10 0x7f61380aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613d99c700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 0x7f61381f21b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f613000efc0 tx=0x7f613000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130009ba0 con 0x7f6138159130 2023-07-06T21:40:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613000f040 con 0x7f6138159130 2023-07-06T21:40:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.662+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61380ae3f0 con 0x7f6138159130 2023-07-06T21:40:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.663+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130014480 con 0x7f6138159130 2023-07-06T21:40:23.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.663+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613814b560 con 0x7f6138159130 2023-07-06T21:40:23.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.664+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61381f3180 con 0x7f6138159130 2023-07-06T21:40:23.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.665+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f61300145e0 con 0x7f6138159130 2023-07-06T21:40:23.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.665+0000 7f612dffb700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f611c06ace0 0x7f611c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:23.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.666+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6130089d80 con 0x7f6138159130 2023-07-06T21:40:23.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.666+0000 7f613d19b700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f611c06ace0 0x7f611c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:23.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.666+0000 7f613d19b700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f611c06ace0 0x7f611c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f61340011e0 tx=0x7f6134009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:23.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.670+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6130054df0 con 0x7f6138159130 2023-07-06T21:40:23.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.879+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f61380ad820 con 0x7f6138159130 2023-07-06T21:40:23.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.879+0000 7f612dffb700 1 -- 172.21.15.84:0/3391427366 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6130018080 con 0x7f6138159130 2023-07-06T21:40:23.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f611c06ace0 msgr2=0x7f611c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f611c06ace0 0x7f611c06d190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f61340011e0 tx=0x7f6134009040 comp rx=0 tx=0).stop 2023-07-06T21:40:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 msgr2=0x7f61381f21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 0x7f61381f21b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f613000efc0 tx=0x7f613000c3a0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 shutdown_connections 2023-07-06T21:40:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f611c06ace0 0x7f611c06d190 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6138159130 0x7f61381f21b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6138070230 0x7f613806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 --2- 172.21.15.84:0/3391427366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61380c9c10 0x7f61380aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:23.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 >> 172.21.15.84:0/3391427366 conn(0x7f61380cf3d0 msgr2=0x7f61380ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:23.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 shutdown_connections 2023-07-06T21:40:23.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:23.882+0000 7f613f3ff700 1 -- 172.21.15.84:0/3391427366 wait complete. 2023-07-06T21:40:23.888 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:23.902 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2023-07-06T21:40:23.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:23.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-07-06T21:40:23.980 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:23.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:40:23.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:23.982 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T21:40:24.039 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:40:24.040 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:24.083 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:40:24.084 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:24.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:24.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.397+0000 7f924a3ff700 1 Processor -- start 2023-07-06T21:40:24.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.397+0000 7f924a3ff700 1 -- start start 2023-07-06T21:40:24.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.398+0000 7f924a3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92440b59c0 0x7f92440ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.398+0000 7f924a3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f92440cf520 0x7f92440cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.398+0000 7f924a3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92440cd820 con 0x7f92440b1f40 2023-07-06T21:40:24.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.398+0000 7f924a3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92440b2310 con 0x7f92440b59c0 2023-07-06T21:40:24.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.398+0000 7f924a3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92440d2710 con 0x7f92440cf520 2023-07-06T21:40:24.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.398+0000 7f92437fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92440b59c0 0x7f92440ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92440b59c0 0x7f92440ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45904/0 (socket says 172.21.15.84:45904) 2023-07-06T21:40:24.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 -- 172.21.15.84:0/2933286701 learned_addr learned my addr 172.21.15.84:0/2933286701 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:24.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f924899c700 1 --2- 172.21.15.84:0/2933286701 >> v2:172.21.15.107:3300/0 conn(0x7f92440cf520 0x7f92440cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 -- 172.21.15.84:0/2933286701 >> v2:172.21.15.107:3300/0 conn(0x7f92440cf520 msgr2=0x7f92440cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:24.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 --2- 172.21.15.84:0/2933286701 >> v2:172.21.15.107:3300/0 conn(0x7f92440cf520 0x7f92440cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 -- 172.21.15.84:0/2933286701 >> v1:172.21.15.107:6789/0 conn(0x7f92440b1f40 legacy=0x7f92440d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 -- 172.21.15.84:0/2933286701 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440cfea0 con 0x7f92440b59c0 2023-07-06T21:40:24.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f9242ffd700 1 -- 172.21.15.84:0/2933286701 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377747349 0 0) 0x7f92440cd820 con 0x7f92440b1f40 2023-07-06T21:40:24.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f92437fe700 1 --2- 172.21.15.84:0/2933286701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440b59c0 0x7f92440ced20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f92440b56e0 tx=0x7f9238009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2613caff2536b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:24.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f9242ffd700 1 -- 172.21.15.84:0/2933286701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011830 con 0x7f92440b59c0 2023-07-06T21:40:24.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f9242ffd700 1 -- 172.21.15.84:0/2933286701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9238011990 con 0x7f92440b59c0 2023-07-06T21:40:24.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.399+0000 7f9242ffd700 1 -- 172.21.15.84:0/2933286701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9238011d20 con 0x7f92440b59c0 2023-07-06T21:40:24.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 -- 172.21.15.84:0/2933286701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440b59c0 msgr2=0x7f92440ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 --2- 172.21.15.84:0/2933286701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440b59c0 0x7f92440ced20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f92440b56e0 tx=0x7f9238009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 -- 172.21.15.84:0/2933286701 shutdown_connections 2023-07-06T21:40:24.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 --2- 172.21.15.84:0/2933286701 >> v2:172.21.15.107:3300/0 conn(0x7f92440cf520 0x7f92440cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 --2- 172.21.15.84:0/2933286701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440b59c0 0x7f92440ced20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 -- 172.21.15.84:0/2933286701 >> 172.21.15.84:0/2933286701 conn(0x7f92440715c0 msgr2=0x7f9244071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:24.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 -- 172.21.15.84:0/2933286701 shutdown_connections 2023-07-06T21:40:24.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.400+0000 7f924a3ff700 1 -- 172.21.15.84:0/2933286701 wait complete. 2023-07-06T21:40:24.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.402+0000 7f924a3ff700 1 Processor -- start 2023-07-06T21:40:24.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.402+0000 7f924a3ff700 1 -- start start 2023-07-06T21:40:24.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924a3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92440b1f40 0x7f92440c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92440b59c0 0x7f924415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924a3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 0x7f92440c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924a3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92440c6df0 con 0x7f92440cf520 2023-07-06T21:40:24.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924a3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92440c70d0 con 0x7f92440b1f40 2023-07-06T21:40:24.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92440c73b0 con 0x7f92440b59c0 2023-07-06T21:40:24.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 0x7f92440c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 0x7f92440c8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45918/0 (socket says 172.21.15.84:45918) 2023-07-06T21:40:24.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f924899c700 1 -- 172.21.15.84:0/1143954089 learned_addr learned my addr 172.21.15.84:0/1143954089 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:24.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f9243fff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92440b1f40 0x7f92440c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.403+0000 7f92437fe700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92440b59c0 0x7f924415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924899c700 1 -- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92440b59c0 msgr2=0x7f924415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924899c700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92440b59c0 0x7f924415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924899c700 1 -- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92440b1f40 msgr2=0x7f92440c6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924899c700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92440b1f40 0x7f92440c6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924899c700 1 -- 172.21.15.84:0/1143954089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9238009550 con 0x7f92440cf520 2023-07-06T21:40:24.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924899c700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 0x7f92440c8be0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f923c00efc0 tx=0x7f923c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c00cd40 con 0x7f92440cf520 2023-07-06T21:40:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c00f040 con 0x7f92440cf520 2023-07-06T21:40:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92440c7690 con 0x7f92440cf520 2023-07-06T21:40:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0146c0 con 0x7f92440cf520 2023-07-06T21:40:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.404+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92440c9930 con 0x7f92440cf520 2023-07-06T21:40:24.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.405+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92440c7bb0 con 0x7f92440cf520 2023-07-06T21:40:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.406+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f923c014820 con 0x7f92440cf520 2023-07-06T21:40:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.407+0000 7f9240ff9700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f923006ace0 0x7f923006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.407+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f923c089e30 con 0x7f92440cf520 2023-07-06T21:40:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.407+0000 7f9243fff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f923006ace0 0x7f923006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.408+0000 7f9243fff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f923006ace0 0x7f923006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9234000f60 tx=0x7f9234009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.412+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92440c7bb0 con 0x7f92440cf520 2023-07-06T21:40:24.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.611+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92440c7bb0 con 0x7f92440cf520 2023-07-06T21:40:24.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.612+0000 7f9240ff9700 1 -- 172.21.15.84:0/1143954089 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f92440c7bb0 con 0x7f92440cf520 2023-07-06T21:40:24.615 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:24.615 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:24.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f923006ace0 msgr2=0x7f923006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f923006ace0 0x7f923006d190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9234000f60 tx=0x7f9234009040 comp rx=0 tx=0).stop 2023-07-06T21:40:24.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 msgr2=0x7f92440c8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 0x7f92440c8be0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f923c00efc0 tx=0x7f923c00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 shutdown_connections 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f923006ace0 0x7f923006d190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92440cf520 0x7f92440c8be0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92440b1f40 0x7f92440c6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 --2- 172.21.15.84:0/1143954089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92440b59c0 0x7f924415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 >> 172.21.15.84:0/1143954089 conn(0x7f92440715c0 msgr2=0x7f92440cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:24.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 shutdown_connections 2023-07-06T21:40:24.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.615+0000 7f924a3ff700 1 -- 172.21.15.84:0/1143954089 wait complete. 2023-07-06T21:40:24.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:24.637 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.832+0000 7fe302fff700 1 Processor -- start 2023-07-06T21:40:24.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.832+0000 7fe302fff700 1 -- start start 2023-07-06T21:40:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.833+0000 7fe302fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe2fc0b81c0 0x7fe2fc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.833+0000 7fe302fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc0d1830 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.833+0000 7fe302fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc160d40 con 0x7fe2fc0b1a80 2023-07-06T21:40:24.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.833+0000 7fe302fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc0cf330 con 0x7fe2fc0b81c0 2023-07-06T21:40:24.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.833+0000 7fe30159c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe2fc0ceda0 0x7fe2fc160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46174/0 (socket says 172.21.15.84:46174) 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.833+0000 7fe30159c700 1 -- 172.21.15.84:0/3936178432 learned_addr learned my addr 172.21.15.84:0/3936178432 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe300d9b700 1 --2- 172.21.15.84:0/3936178432 >> v2:172.21.15.107:3300/0 conn(0x7fe2fc0b81c0 0x7fe2fc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870303692 0 0) 0x7fe2fc0d1830 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2ec0035b0 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637699884 0 0) 0x7fe2fc160d40 con 0x7fe2fc0b1a80 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2fc0d1830 con 0x7fe2fc0b1a80 2023-07-06T21:40:24.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe300d9b700 1 -- 172.21.15.84:0/3936178432 >> v1:172.21.15.107:6789/0 conn(0x7fe2fc0b1a80 legacy=0x7fe2fc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe300d9b700 1 -- 172.21.15.84:0/3936178432 >> v1:172.21.15.84:6789/0 conn(0x7fe2fc0ceda0 legacy=0x7fe2fc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.834+0000 7fe300d9b700 1 -- 172.21.15.84:0/3936178432 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc0d1270 con 0x7fe2fc0b81c0 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe300d9b700 1 --2- 172.21.15.84:0/3936178432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc0cf600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009840 tx=0x7fe2f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff771372c763506c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0011830 con 0x7fe2fc0b81c0 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f0011990 con 0x7fe2fc0b81c0 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe2fb7fe700 1 -- 172.21.15.84:0/3936178432 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0011d20 con 0x7fe2fc0b81c0 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe302fff700 1 -- 172.21.15.84:0/3936178432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 msgr2=0x7fe2fc0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe302fff700 1 --2- 172.21.15.84:0/3936178432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc0cf600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009840 tx=0x7fe2f0009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:24.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe302fff700 1 -- 172.21.15.84:0/3936178432 shutdown_connections 2023-07-06T21:40:24.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe302fff700 1 --2- 172.21.15.84:0/3936178432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc0cf600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe302fff700 1 -- 172.21.15.84:0/3936178432 >> 172.21.15.84:0/3936178432 conn(0x7fe2fc06b2b0 msgr2=0x7fe2fc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:24.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.835+0000 7fe302fff700 1 -- 172.21.15.84:0/3936178432 shutdown_connections 2023-07-06T21:40:24.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.836+0000 7fe302fff700 1 -- 172.21.15.84:0/3936178432 wait complete. 2023-07-06T21:40:24.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.837+0000 7fe302fff700 1 Processor -- start 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.837+0000 7fe302fff700 1 -- start start 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.838+0000 7fe302fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2fc0b1a80 0x7fe2fc1f1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.838+0000 7fe302fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.838+0000 7fe302fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 0x7fe2fc0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.838+0000 7fe302fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1e7b80 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.838+0000 7fe302fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1e7e60 con 0x7fe2fc0b81c0 2023-07-06T21:40:24.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.838+0000 7fe302fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1e8140 con 0x7fe2fc0b1a80 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 0x7fe2fc0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe300d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2fc0b1a80 0x7fe2fc1f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 0x7fe2fc0c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45928/0 (socket says 172.21.15.84:45928) 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 -- 172.21.15.84:0/2756634813 learned_addr learned my addr 172.21.15.84:0/2756634813 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe2fbfff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 -- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2fc0b1a80 msgr2=0x7fe2fc1f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2fc0b1a80 0x7fe2fc1f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 -- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 msgr2=0x7fe2fc159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 -- 172.21.15.84:0/2756634813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f00094a0 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.839+0000 7fe30159c700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 0x7fe2fc0c7260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400caa0 tx=0x7fe2f400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.840+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f400a9e0 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.840+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f400ab60 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.840+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4012400 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.840+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc1e8420 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.840+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc1e7800 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.841+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc1e8940 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.842+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe2f401b3b0 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.843+0000 7fe2f97fa700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe2e006ace0 0x7fe2e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:24.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.843+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe2f4089050 con 0x7fe2fc0ceda0 2023-07-06T21:40:24.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.843+0000 7fe300d9b700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe2e006ace0 0x7fe2e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:24.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.844+0000 7fe300d9b700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe2e006ace0 0x7fe2e006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009fd0 tx=0x7fe2f001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:24.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:24.847+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe2fc1e8940 con 0x7fe2fc0ceda0 2023-07-06T21:40:25.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.045+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2fc1e8940 con 0x7fe2fc0ceda0 2023-07-06T21:40:25.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.045+0000 7fe2f97fa700 1 -- 172.21.15.84:0/2756634813 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2f4017020 con 0x7fe2fc0ceda0 2023-07-06T21:40:25.051 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:25.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe2e006ace0 msgr2=0x7fe2e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe2e006ace0 0x7fe2e006d190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0009fd0 tx=0x7fe2f001c040 comp rx=0 tx=0).stop 2023-07-06T21:40:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 msgr2=0x7fe2fc0c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 0x7fe2fc0c7260 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400caa0 tx=0x7fe2f400cdb0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 shutdown_connections 2023-07-06T21:40:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe2e006ace0 0x7fe2e006d190 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2fc0ceda0 0x7fe2fc0c7260 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2fc0b1a80 0x7fe2fc1f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 --2- 172.21.15.84:0/2756634813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2fc0b81c0 0x7fe2fc159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 >> 172.21.15.84:0/2756634813 conn(0x7fe2fc06b2b0 msgr2=0x7fe2fc15c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 shutdown_connections 2023-07-06T21:40:25.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.048+0000 7fe302fff700 1 -- 172.21.15.84:0/2756634813 wait complete. 2023-07-06T21:40:25.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:25.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.262+0000 7fe0a4bff700 1 Processor -- start 2023-07-06T21:40:25.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.263+0000 7fe0a4bff700 1 -- start start 2023-07-06T21:40:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe0a4bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe0a00b81c0 0x7fe0a00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe0a4bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0a00b1a80 0x7fe0a00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe0a4bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0a00d1830 con 0x7fe0a00ceda0 2023-07-06T21:40:25.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe0a4bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a00cf330 con 0x7fe0a00b1a80 2023-07-06T21:40:25.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe0a4bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a00d1270 con 0x7fe0a00b81c0 2023-07-06T21:40:25.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe097fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0a00b1a80 0x7fe0a00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe097fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe0a00b1a80 0x7fe0a00cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45934/0 (socket says 172.21.15.84:45934) 2023-07-06T21:40:25.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe097fff700 1 -- 172.21.15.84:0/3062645184 learned_addr learned my addr 172.21.15.84:0/3062645184 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:25.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.264+0000 7fe09e59c700 1 --2- 172.21.15.84:0/3062645184 >> v2:172.21.15.107:3300/0 conn(0x7fe0a00b81c0 0x7fe0a00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe097fff700 1 -- 172.21.15.84:0/3062645184 >> v2:172.21.15.107:3300/0 conn(0x7fe0a00b81c0 msgr2=0x7fe0a00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe097fff700 1 --2- 172.21.15.84:0/3062645184 >> v2:172.21.15.107:3300/0 conn(0x7fe0a00b81c0 0x7fe0a00cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe097fff700 1 -- 172.21.15.84:0/3062645184 >> v1:172.21.15.107:6789/0 conn(0x7fe0a00ceda0 legacy=0x7fe0a015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe097fff700 1 -- 172.21.15.84:0/3062645184 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00d1550 con 0x7fe0a00b1a80 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe097fff700 1 --2- 172.21.15.84:0/3062645184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00b1a80 0x7fe0a00cfb40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00b2620 tx=0x7fe088009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c914a7fbb24996 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe09dd9b700 1 -- 172.21.15.84:0/3062645184 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088011830 con 0x7fe0a00b1a80 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe09dd9b700 1 -- 172.21.15.84:0/3062645184 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe088011990 con 0x7fe0a00b1a80 2023-07-06T21:40:25.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.265+0000 7fe09dd9b700 1 -- 172.21.15.84:0/3062645184 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe088011d20 con 0x7fe0a00b1a80 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 -- 172.21.15.84:0/3062645184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00b1a80 msgr2=0x7fe0a00cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/3062645184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00b1a80 0x7fe0a00cfb40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00b2620 tx=0x7fe088009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 -- 172.21.15.84:0/3062645184 shutdown_connections 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/3062645184 >> v2:172.21.15.107:3300/0 conn(0x7fe0a00b81c0 0x7fe0a00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/3062645184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00b1a80 0x7fe0a00cfb40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 -- 172.21.15.84:0/3062645184 >> 172.21.15.84:0/3062645184 conn(0x7fe0a006b2b0 msgr2=0x7fe0a006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:25.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 -- 172.21.15.84:0/3062645184 shutdown_connections 2023-07-06T21:40:25.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.266+0000 7fe0a4bff700 1 -- 172.21.15.84:0/3062645184 wait complete. 2023-07-06T21:40:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.267+0000 7fe0a4bff700 1 Processor -- start 2023-07-06T21:40:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 -- start start 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0a00b1a80 0x7fe0a0153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a00b81c0 0x7fe0a0152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 0x7fe0a00c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00c7d20 con 0x7fe0a00ceda0 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00c8000 con 0x7fe0a00b81c0 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.268+0000 7fe0a4bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a00c82e0 con 0x7fe0a00b1a80 2023-07-06T21:40:25.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 0x7fe0a00c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 0x7fe0a00c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45942/0 (socket says 172.21.15.84:45942) 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 -- 172.21.15.84:0/2105702745 learned_addr learned my addr 172.21.15.84:0/2105702745 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09e59c700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0a00b1a80 0x7fe0a0153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe097fff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a00b81c0 0x7fe0a0152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 -- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0a00b1a80 msgr2=0x7fe0a0153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0a00b1a80 0x7fe0a0153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 -- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a00b81c0 msgr2=0x7fe0a0152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a00b81c0 0x7fe0a0152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 -- 172.21.15.84:0/2105702745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088009550 con 0x7fe0a00ceda0 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.269+0000 7fe09ed9d700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 0x7fe0a00c90a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe09800cc50 tx=0x7fe09800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.270+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09800ad80 con 0x7fe0a00ceda0 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.270+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09800ce50 con 0x7fe0a00ceda0 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.270+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a00c85c0 con 0x7fe0a00ceda0 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.270+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe098003c40 con 0x7fe0a00ceda0 2023-07-06T21:40:25.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.270+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a00c8b80 con 0x7fe0a00ceda0 2023-07-06T21:40:25.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.271+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a00c8da0 con 0x7fe0a00ceda0 2023-07-06T21:40:25.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.272+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe0980050d0 con 0x7fe0a00ceda0 2023-07-06T21:40:25.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.272+0000 7fe096ffd700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe08006ace0 0x7fe08006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.273+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe098089630 con 0x7fe0a00ceda0 2023-07-06T21:40:25.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.273+0000 7fe09e59c700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe08006ace0 0x7fe08006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.273+0000 7fe09e59c700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe08006ace0 0x7fe08006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00c7850 tx=0x7fe090008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:25.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.277+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe098054600 con 0x7fe0a00ceda0 2023-07-06T21:40:25.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.471+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe0a004e1d0 con 0x7fe0a00ceda0 2023-07-06T21:40:25.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.471+0000 7fe096ffd700 1 -- 172.21.15.84:0/2105702745 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe098057c20 con 0x7fe0a00ceda0 2023-07-06T21:40:25.474 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:25.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.473+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe08006ace0 msgr2=0x7fe08006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.473+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe08006ace0 0x7fe08006d190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00c7850 tx=0x7fe090008040 comp rx=0 tx=0).stop 2023-07-06T21:40:25.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.473+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 msgr2=0x7fe0a00c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.473+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 0x7fe0a00c90a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe09800cc50 tx=0x7fe09800a340 comp rx=0 tx=0).stop 2023-07-06T21:40:25.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 shutdown_connections 2023-07-06T21:40:25.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe08006ace0 0x7fe08006d190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe0a00ceda0 0x7fe0a00c90a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0a00b1a80 0x7fe0a0153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 --2- 172.21.15.84:0/2105702745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0a00b81c0 0x7fe0a0152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 >> 172.21.15.84:0/2105702745 conn(0x7fe0a006b2b0 msgr2=0x7fe0a00732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:25.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 shutdown_connections 2023-07-06T21:40:25.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.474+0000 7fe0a4bff700 1 -- 172.21.15.84:0/2105702745 wait complete. 2023-07-06T21:40:25.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:25.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.687+0000 7f68f7fff700 1 Processor -- start 2023-07-06T21:40:25.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.687+0000 7f68f7fff700 1 -- start start 2023-07-06T21:40:25.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f7fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f68f00b81c0 0x7f68f00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f7fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68f00ceda0 0x7f68f00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f7fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00d1830 con 0x7f68f00b1a80 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f7fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f00cf330 con 0x7f68f00ceda0 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f7fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f00d1270 con 0x7f68f00b81c0 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68f00ceda0 0x7f68f00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f68f00ceda0 0x7f68f00cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45954/0 (socket says 172.21.15.84:45954) 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.688+0000 7f68f659c700 1 -- 172.21.15.84:0/2309474306 learned_addr learned my addr 172.21.15.84:0/2309474306 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:25.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f5d9b700 1 --2- 172.21.15.84:0/2309474306 >> v2:172.21.15.107:3300/0 conn(0x7f68f00b81c0 0x7f68f00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f659c700 1 -- 172.21.15.84:0/2309474306 >> v2:172.21.15.107:3300/0 conn(0x7f68f00b81c0 msgr2=0x7f68f00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f659c700 1 --2- 172.21.15.84:0/2309474306 >> v2:172.21.15.107:3300/0 conn(0x7f68f00b81c0 0x7f68f00cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f659c700 1 -- 172.21.15.84:0/2309474306 >> v1:172.21.15.107:6789/0 conn(0x7f68f00b1a80 legacy=0x7f68f015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f659c700 1 -- 172.21.15.84:0/2309474306 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00d1550 con 0x7f68f00ceda0 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f5d9b700 1 --2- 172.21.15.84:0/2309474306 >> v2:172.21.15.107:3300/0 conn(0x7f68f00b81c0 0x7f68f00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f659c700 1 --2- 172.21.15.84:0/2309474306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f00cfb40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f68f00b2620 tx=0x7f68e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267385483a9ca0b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:25.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f4d99700 1 -- 172.21.15.84:0/2309474306 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e800e070 con 0x7f68f00ceda0 2023-07-06T21:40:25.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f4d99700 1 -- 172.21.15.84:0/2309474306 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e8003a20 con 0x7f68f00ceda0 2023-07-06T21:40:25.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.689+0000 7f68f4d99700 1 -- 172.21.15.84:0/2309474306 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8003dd0 con 0x7f68f00ceda0 2023-07-06T21:40:25.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 -- 172.21.15.84:0/2309474306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 msgr2=0x7f68f00cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 --2- 172.21.15.84:0/2309474306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f00cfb40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f68f00b2620 tx=0x7f68e800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 -- 172.21.15.84:0/2309474306 shutdown_connections 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 --2- 172.21.15.84:0/2309474306 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f00cfb40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 --2- 172.21.15.84:0/2309474306 >> v2:172.21.15.107:3300/0 conn(0x7f68f00b81c0 0x7f68f00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 -- 172.21.15.84:0/2309474306 >> 172.21.15.84:0/2309474306 conn(0x7f68f006b2b0 msgr2=0x7f68f006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 -- 172.21.15.84:0/2309474306 shutdown_connections 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.690+0000 7f68f7fff700 1 -- 172.21.15.84:0/2309474306 wait complete. 2023-07-06T21:40:25.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.691+0000 7f68f7fff700 1 Processor -- start 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.692+0000 7f68f7fff700 1 -- start start 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.692+0000 7f68f7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68f00b1a80 0x7f68f01f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.692+0000 7f68f7fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68f00b81c0 0x7f68f00a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.692+0000 7f68f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f7fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f01f5120 con 0x7f68f00ceda0 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f7fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f0158150 con 0x7f68f00b1a80 2023-07-06T21:40:25.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f7fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00cad80 con 0x7f68f00b81c0 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45956/0 (socket says 172.21.15.84:45956) 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 -- 172.21.15.84:0/3859121552 learned_addr learned my addr 172.21.15.84:0/3859121552 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f5d9b700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68f00b1a80 0x7f68f01f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f559a700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68f00b81c0 0x7f68f00a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 -- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68f00b81c0 msgr2=0x7f68f00a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68f00b81c0 0x7f68f00a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 -- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68f00b1a80 msgr2=0x7f68f01f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68f00b1a80 0x7f68f01f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 -- 172.21.15.84:0/3859121552 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8009b30 con 0x7f68f00ceda0 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.693+0000 7f68f659c700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f015a650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68e8015040 tx=0x7f68e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.694+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8004240 con 0x7f68f00ceda0 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.694+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80043a0 con 0x7f68f00ceda0 2023-07-06T21:40:25.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.694+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8004750 con 0x7f68f00ceda0 2023-07-06T21:40:25.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.694+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f00cb060 con 0x7f68f00ceda0 2023-07-06T21:40:25.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.694+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f01586d0 con 0x7f68f00ceda0 2023-07-06T21:40:25.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.695+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f004e1d0 con 0x7f68f00ceda0 2023-07-06T21:40:25.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.696+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f68e8019070 con 0x7f68f00ceda0 2023-07-06T21:40:25.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.697+0000 7f68e67fc700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f68d406ace0 0x7f68d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:25.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.697+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f68e808a4c0 con 0x7f68f00ceda0 2023-07-06T21:40:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.697+0000 7f68f5d9b700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f68d406ace0 0x7f68d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.697+0000 7f68f5d9b700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f68d406ace0 0x7f68d406d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:25.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.701+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f68e8055520 con 0x7f68f00ceda0 2023-07-06T21:40:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.896+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68f00c6070 con 0x7f68f00ceda0 2023-07-06T21:40:25.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.896+0000 7f68e67fc700 1 -- 172.21.15.84:0/3859121552 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68e8058b40 con 0x7f68f00ceda0 2023-07-06T21:40:25.900 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f68d406ace0 msgr2=0x7f68d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f68d406ace0 0x7f68d406d190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f68ec006fd0 tx=0x7f68ec008040 comp rx=0 tx=0).stop 2023-07-06T21:40:25.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 msgr2=0x7f68f015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f015a650 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f68e8015040 tx=0x7f68e8004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 shutdown_connections 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f68d406ace0 0x7f68d406d190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68f00ceda0 0x7f68f015a650 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68f00b1a80 0x7f68f01f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 --2- 172.21.15.84:0/3859121552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68f00b81c0 0x7f68f00a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 >> 172.21.15.84:0/3859121552 conn(0x7f68f006b2b0 msgr2=0x7f68f00cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:25.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 shutdown_connections 2023-07-06T21:40:25.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:25.899+0000 7f68f7fff700 1 -- 172.21.15.84:0/3859121552 wait complete. 2023-07-06T21:40:25.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:26.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.115+0000 7f65ecdff700 1 Processor -- start 2023-07-06T21:40:26.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.116+0000 7f65ecdff700 1 -- start start 2023-07-06T21:40:26.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.117+0000 7f65ecdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 0x7f65e80c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.117+0000 7f65ecdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c460 0x7f65e8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.117+0000 7f65ecdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e806cf40 con 0x7f65e815c8d0 2023-07-06T21:40:26.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.117+0000 7f65ecdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e815d400 con 0x7f65e815c020 2023-07-06T21:40:26.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.117+0000 7f65ecdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e815ce40 con 0x7f65e815c460 2023-07-06T21:40:26.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e5d9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c460 0x7f65e8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e659c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 0x7f65e80c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e659c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 0x7f65e80c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:36634/0 (socket says 172.21.15.84:36634) 2023-07-06T21:40:26.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e659c700 1 -- 172.21.15.84:0/1096863481 learned_addr learned my addr 172.21.15.84:0/1096863481 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:26.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e6d9d700 1 --1- 172.21.15.84:0/1096863481 >> v1:172.21.15.107:6789/0 conn(0x7f65e815c8d0 0x7f65e80703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:43290/0 (socket says 172.21.15.84:43290) 2023-07-06T21:40:26.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e5d9b700 1 -- 172.21.15.84:0/1096863481 >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 msgr2=0x7f65e80c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:26.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e5d9b700 1 --2- 172.21.15.84:0/1096863481 >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 0x7f65e80c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e5d9b700 1 -- 172.21.15.84:0/1096863481 >> v1:172.21.15.107:6789/0 conn(0x7f65e815c8d0 legacy=0x7f65e80703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e5d9b700 1 -- 172.21.15.84:0/1096863481 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e815d120 con 0x7f65e815c460 2023-07-06T21:40:26.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e659c700 1 --2- 172.21.15.84:0/1096863481 >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 0x7f65e80c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:26.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.118+0000 7f65e559a700 1 -- 172.21.15.84:0/1096863481 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3470138098 0 0) 0x7f65e806cf40 con 0x7f65e815c8d0 2023-07-06T21:40:26.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.119+0000 7f65e5d9b700 1 --2- 172.21.15.84:0/1096863481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c460 0x7f65e8069e90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009840 tx=0x7f65dc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3529ceb1c3168a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:26.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.119+0000 7f65e559a700 1 -- 172.21.15.84:0/1096863481 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011830 con 0x7f65e815c460 2023-07-06T21:40:26.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.119+0000 7f65e559a700 1 -- 172.21.15.84:0/1096863481 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc011990 con 0x7f65e815c460 2023-07-06T21:40:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.119+0000 7f65e559a700 1 -- 172.21.15.84:0/1096863481 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc011d20 con 0x7f65e815c460 2023-07-06T21:40:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.119+0000 7f65ecdff700 1 -- 172.21.15.84:0/1096863481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c460 msgr2=0x7f65e8069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.119+0000 7f65ecdff700 1 --2- 172.21.15.84:0/1096863481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c460 0x7f65e8069e90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009840 tx=0x7f65dc009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.120+0000 7f65ecdff700 1 -- 172.21.15.84:0/1096863481 shutdown_connections 2023-07-06T21:40:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.120+0000 7f65ecdff700 1 --2- 172.21.15.84:0/1096863481 >> v2:172.21.15.107:3300/0 conn(0x7f65e815c020 0x7f65e80c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.120+0000 7f65ecdff700 1 --2- 172.21.15.84:0/1096863481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c460 0x7f65e8069e90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.120+0000 7f65ecdff700 1 -- 172.21.15.84:0/1096863481 >> 172.21.15.84:0/1096863481 conn(0x7f65e80cbf10 msgr2=0x7f65e80ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:26.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.120+0000 7f65ecdff700 1 -- 172.21.15.84:0/1096863481 shutdown_connections 2023-07-06T21:40:26.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.120+0000 7f65ecdff700 1 -- 172.21.15.84:0/1096863481 wait complete. 2023-07-06T21:40:26.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.121+0000 7f65ecdff700 1 Processor -- start 2023-07-06T21:40:26.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.121+0000 7f65ecdff700 1 -- start start 2023-07-06T21:40:26.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.122+0000 7f65ecdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 0x7f65e814bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.122+0000 7f65ecdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e815c460 0x7f65e80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.122+0000 7f65ecdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c8d0 0x7f65e80af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.122+0000 7f65ecdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e814c440 con 0x7f65e815c020 2023-07-06T21:40:26.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.122+0000 7f65ecdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80ac890 con 0x7f65e815c460 2023-07-06T21:40:26.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.122+0000 7f65ecdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80afa80 con 0x7f65e815c8d0 2023-07-06T21:40:26.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 0x7f65e814bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 0x7f65e814bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45960/0 (socket says 172.21.15.84:45960) 2023-07-06T21:40:26.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 -- 172.21.15.84:0/283304727 learned_addr learned my addr 172.21.15.84:0/283304727 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:26.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e6d9d700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c8d0 0x7f65e80af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e5d9b700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e815c460 0x7f65e80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 -- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c8d0 msgr2=0x7f65e80af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c8d0 0x7f65e80af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 -- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e815c460 msgr2=0x7f65e80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e815c460 0x7f65e80ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 -- 172.21.15.84:0/283304727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d800b7f0 con 0x7f65e815c020 2023-07-06T21:40:26.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.123+0000 7f65e659c700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 0x7f65e814bf00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65d800dd20 tx=0x7f65d8009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:26.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.124+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d800de10 con 0x7f65e815c020 2023-07-06T21:40:26.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.124+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d8009b70 con 0x7f65e815c020 2023-07-06T21:40:26.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.124+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc0094a0 con 0x7f65e815c020 2023-07-06T21:40:26.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.124+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8017410 con 0x7f65e815c020 2023-07-06T21:40:26.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.124+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e80b0000 con 0x7f65e815c020 2023-07-06T21:40:26.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.125+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e804e1d0 con 0x7f65e815c020 2023-07-06T21:40:26.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.126+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65d8017570 con 0x7f65e815c020 2023-07-06T21:40:26.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.126+0000 7f65d6ffd700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c806ace0 0x7f65c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.127+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f65d800b050 con 0x7f65e815c020 2023-07-06T21:40:26.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.127+0000 7f65e5d9b700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c806ace0 0x7f65c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.127+0000 7f65e5d9b700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c806ace0 0x7f65c806d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0090b0 tx=0x7f65dc01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:26.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.131+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65d8054430 con 0x7f65e815c020 2023-07-06T21:40:26.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.326+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65e80b4780 con 0x7f65e815c020 2023-07-06T21:40:26.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.327+0000 7f65d6ffd700 1 -- 172.21.15.84:0/283304727 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65d8057a50 con 0x7f65e815c020 2023-07-06T21:40:26.330 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:26.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c806ace0 msgr2=0x7f65c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c806ace0 0x7f65c806d190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f65dc0090b0 tx=0x7f65dc01c040 comp rx=0 tx=0).stop 2023-07-06T21:40:26.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 msgr2=0x7f65e814bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 0x7f65e814bf00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65d800dd20 tx=0x7f65d8009400 comp rx=0 tx=0).stop 2023-07-06T21:40:26.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 shutdown_connections 2023-07-06T21:40:26.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c806ace0 0x7f65c806d190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65e815c8d0 0x7f65e80af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65e815c020 0x7f65e814bf00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 --2- 172.21.15.84:0/283304727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65e815c460 0x7f65e80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 >> 172.21.15.84:0/283304727 conn(0x7f65e80cbf10 msgr2=0x7f65e814c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:26.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 shutdown_connections 2023-07-06T21:40:26.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.329+0000 7f65ecdff700 1 -- 172.21.15.84:0/283304727 wait complete. 2023-07-06T21:40:26.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:26.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.541+0000 7f9a6e5ff700 1 Processor -- start 2023-07-06T21:40:26.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.541+0000 7f9a6e5ff700 1 -- start start 2023-07-06T21:40:26.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.542+0000 7f9a6e5ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a680b83c0 0x7f9a680cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.542+0000 7f9a6e5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a680cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.542+0000 7f9a6e5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a680cfee0 con 0x7f9a680cfa90 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.542+0000 7f9a6e5ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a680d0a80 con 0x7f9a680b83c0 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.542+0000 7f9a6e5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a680b8790 con 0x7f9a680d3710 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.542+0000 7f9a67fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a680b83c0 0x7f9a680cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9a680b83c0 0x7f9a680cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45972/0 (socket says 172.21.15.84:45972) 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 -- 172.21.15.84:0/2633292406 learned_addr learned my addr 172.21.15.84:0/2633292406 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:26.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a677fe700 1 --2- 172.21.15.84:0/2633292406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a680cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 -- 172.21.15.84:0/2633292406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 msgr2=0x7f9a680cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:26.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 --2- 172.21.15.84:0/2633292406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a680cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 -- 172.21.15.84:0/2633292406 >> v1:172.21.15.107:6789/0 conn(0x7f9a680cfa90 legacy=0x7f9a6815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 -- 172.21.15.84:0/2633292406 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a680b4e70 con 0x7f9a680b83c0 2023-07-06T21:40:26.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a67fff700 1 --2- 172.21.15.84:0/2633292406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680b83c0 0x7f9a680cd200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9a580098f0 tx=0x7f9a58009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e082d20f57521dc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:26.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.543+0000 7f9a66ffd700 1 -- 172.21.15.84:0/2633292406 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a58011830 con 0x7f9a680b83c0 2023-07-06T21:40:26.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a66ffd700 1 -- 172.21.15.84:0/2633292406 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a58011990 con 0x7f9a680b83c0 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a66ffd700 1 -- 172.21.15.84:0/2633292406 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a58011d20 con 0x7f9a680b83c0 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/2633292406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680b83c0 msgr2=0x7f9a680cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/2633292406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680b83c0 0x7f9a680cd200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9a580098f0 tx=0x7f9a58009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/2633292406 shutdown_connections 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/2633292406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680b83c0 0x7f9a680cd200 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/2633292406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a680cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/2633292406 >> 172.21.15.84:0/2633292406 conn(0x7f9a6806b350 msgr2=0x7f9a6806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:26.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/2633292406 shutdown_connections 2023-07-06T21:40:26.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.544+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/2633292406 wait complete. 2023-07-06T21:40:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.546+0000 7f9a6e5ff700 1 Processor -- start 2023-07-06T21:40:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.546+0000 7f9a6e5ff700 1 -- start start 2023-07-06T21:40:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a6e5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a680b83c0 0x7f9a68154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a6e5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 0x7f9a680cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a6e5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a68149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a6e5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6815dee0 con 0x7f9a680cfa90 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a6e5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6815e130 con 0x7f9a680b83c0 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a6e5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6815e410 con 0x7f9a680d3710 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 0x7f9a680cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 0x7f9a680cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45982/0 (socket says 172.21.15.84:45982) 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a677fe700 1 -- 172.21.15.84:0/624680965 learned_addr learned my addr 172.21.15.84:0/624680965 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.547+0000 7f9a67fff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a680b83c0 0x7f9a68154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a6cb9c700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a68149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a677fe700 1 -- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 msgr2=0x7f9a68149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a677fe700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a68149ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a677fe700 1 -- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a680b83c0 msgr2=0x7f9a68154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a677fe700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a680b83c0 0x7f9a68154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a677fe700 1 -- 172.21.15.84:0/624680965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58009550 con 0x7f9a680cfa90 2023-07-06T21:40:26.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a677fe700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 0x7f9a680cafb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00e840 tx=0x7f9a5c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c009590 con 0x7f9a680cfa90 2023-07-06T21:40:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c013680 con 0x7f9a680cfa90 2023-07-06T21:40:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6815e6f0 con 0x7f9a680cfa90 2023-07-06T21:40:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c013a30 con 0x7f9a680cfa90 2023-07-06T21:40:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.548+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6815ecb0 con 0x7f9a680cfa90 2023-07-06T21:40:26.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.549+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a680b6ac0 con 0x7f9a680cfa90 2023-07-06T21:40:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.550+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9a5c013b90 con 0x7f9a680cfa90 2023-07-06T21:40:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.551+0000 7f9a64ff9700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9a5406ace0 0x7f9a5406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:26.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.551+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9a5c089380 con 0x7f9a680cfa90 2023-07-06T21:40:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.552+0000 7f9a67fff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9a5406ace0 0x7f9a5406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:26.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.552+0000 7f9a67fff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9a5406ace0 0x7f9a5406d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9a58014040 tx=0x7f9a58008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:26.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.556+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9a5c057de0 con 0x7f9a680cfa90 2023-07-06T21:40:26.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.768+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9a680b67e0 con 0x7f9a680cfa90 2023-07-06T21:40:26.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.768+0000 7f9a64ff9700 1 -- 172.21.15.84:0/624680965 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9a5c057970 con 0x7f9a680cfa90 2023-07-06T21:40:26.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9a5406ace0 msgr2=0x7f9a5406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9a5406ace0 0x7f9a5406d190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9a58014040 tx=0x7f9a58008030 comp rx=0 tx=0).stop 2023-07-06T21:40:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 msgr2=0x7f9a680cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 0x7f9a680cafb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c00e840 tx=0x7f9a5c00eb50 comp rx=0 tx=0).stop 2023-07-06T21:40:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 shutdown_connections 2023-07-06T21:40:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9a5406ace0 0x7f9a5406d190 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a680d3710 0x7f9a68149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a680b83c0 0x7f9a68154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 --2- 172.21.15.84:0/624680965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a680cfa90 0x7f9a680cafb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:26.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.770+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 >> 172.21.15.84:0/624680965 conn(0x7f9a6806b350 msgr2=0x7f9a6815ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:26.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.771+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 shutdown_connections 2023-07-06T21:40:26.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:26.771+0000 7f9a6e5ff700 1 -- 172.21.15.84:0/624680965 wait complete. 2023-07-06T21:40:26.775 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:26.792 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:40:26.792 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:26.835 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:27.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.064+0000 7ffb46bff700 1 Processor -- start 2023-07-06T21:40:27.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.065+0000 7ffb46bff700 1 -- start start 2023-07-06T21:40:27.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb46bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40159130 0x7ffb400c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb46bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffb40070940 0x7ffb400c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb46bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4015d930 con 0x7ffb400c5000 2023-07-06T21:40:27.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb46bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4015cd20 con 0x7ffb40070940 2023-07-06T21:40:27.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb46bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4015d000 con 0x7ffb40159130 2023-07-06T21:40:27.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb3ffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb400c5000 0x7ffb40070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46190/0 (socket says 172.21.15.84:46190) 2023-07-06T21:40:27.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb3ffff700 1 -- 172.21.15.84:0/2233529251 learned_addr learned my addr 172.21.15.84:0/2233529251 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:27.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb4519c700 1 --2- 172.21.15.84:0/2233529251 >> v2:172.21.15.107:3300/0 conn(0x7ffb40070940 0x7ffb400c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb4499b700 1 --2- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40159130 0x7ffb400c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.066+0000 7ffb3f7fe700 1 -- 172.21.15.84:0/2233529251 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 751462351 0 0) 0x7ffb4015d930 con 0x7ffb400c5000 2023-07-06T21:40:27.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb3f7fe700 1 -- 172.21.15.84:0/2233529251 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb300035b0 con 0x7ffb400c5000 2023-07-06T21:40:27.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb4519c700 1 -- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40159130 msgr2=0x7ffb400c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:27.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb4519c700 1 --2- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40159130 0x7ffb400c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb4519c700 1 -- 172.21.15.84:0/2233529251 >> v1:172.21.15.84:6789/0 conn(0x7ffb400c5000 legacy=0x7ffb40070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb4519c700 1 -- 172.21.15.84:0/2233529251 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4015d2e0 con 0x7ffb40070940 2023-07-06T21:40:27.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb4519c700 1 --2- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb40070940 0x7ffb400c9c10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb3800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a22a51f2888885 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb3f7fe700 1 -- 172.21.15.84:0/2233529251 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800e070 con 0x7ffb40070940 2023-07-06T21:40:27.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb3f7fe700 1 -- 172.21.15.84:0/2233529251 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3800bb20 con 0x7ffb40070940 2023-07-06T21:40:27.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.067+0000 7ffb3f7fe700 1 -- 172.21.15.84:0/2233529251 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800be90 con 0x7ffb40070940 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 -- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb40070940 msgr2=0x7ffb400c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb40070940 0x7ffb400c9c10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ffb38009d90 tx=0x7ffb3800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 -- 172.21.15.84:0/2233529251 shutdown_connections 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb40070940 0x7ffb400c9c10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2233529251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40159130 0x7ffb400c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 -- 172.21.15.84:0/2233529251 >> 172.21.15.84:0/2233529251 conn(0x7ffb400cf3d0 msgr2=0x7ffb400cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:27.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 -- 172.21.15.84:0/2233529251 shutdown_connections 2023-07-06T21:40:27.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.068+0000 7ffb46bff700 1 -- 172.21.15.84:0/2233529251 wait complete. 2023-07-06T21:40:27.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.070+0000 7ffb46bff700 1 Processor -- start 2023-07-06T21:40:27.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.070+0000 7ffb46bff700 1 -- start start 2023-07-06T21:40:27.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb46bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40070940 0x7ffb401f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb46bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb400c5000 0x7ffb4015a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb46bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 0x7ffb401f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb46bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4015a5a0 con 0x7ffb40159130 2023-07-06T21:40:27.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb46bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40159680 con 0x7ffb400c5000 2023-07-06T21:40:27.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb46bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40159960 con 0x7ffb40070940 2023-07-06T21:40:27.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 0x7ffb401f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 0x7ffb401f58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45986/0 (socket says 172.21.15.84:45986) 2023-07-06T21:40:27.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 -- 172.21.15.84:0/2195217266 learned_addr learned my addr 172.21.15.84:0/2195217266 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:27.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4499b700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40070940 0x7ffb401f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb3ffff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb400c5000 0x7ffb4015a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 -- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40070940 msgr2=0x7ffb401f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40070940 0x7ffb401f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 -- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb400c5000 msgr2=0x7ffb4015a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb400c5000 0x7ffb4015a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.071+0000 7ffb4519c700 1 -- 172.21.15.84:0/2195217266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb38009a70 con 0x7ffb40159130 2023-07-06T21:40:27.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.072+0000 7ffb4519c700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 0x7ffb401f58d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb38014040 tx=0x7ffb38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.072+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb38004960 con 0x7ffb40159130 2023-07-06T21:40:27.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.072+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb38020070 con 0x7ffb40159130 2023-07-06T21:40:27.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.072+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb40159c40 con 0x7ffb40159130 2023-07-06T21:40:27.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.072+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb380276c0 con 0x7ffb40159130 2023-07-06T21:40:27.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.072+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb401f5e10 con 0x7ffb40159130 2023-07-06T21:40:27.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.073+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb401f5fa0 con 0x7ffb40159130 2023-07-06T21:40:27.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.074+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb38012430 con 0x7ffb40159130 2023-07-06T21:40:27.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.075+0000 7ffb3d7fa700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb2406ace0 0x7ffb2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.075+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ffb38022070 con 0x7ffb40159130 2023-07-06T21:40:27.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.075+0000 7ffb4499b700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb2406ace0 0x7ffb2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.076+0000 7ffb4499b700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb2406ace0 0x7ffb2406d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009ab0 tx=0x7ffb34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.079+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffb38056670 con 0x7ffb40159130 2023-07-06T21:40:27.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.280+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb40159e30 con 0x7ffb40159130 2023-07-06T21:40:27.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.281+0000 7ffb3d7fa700 1 -- 172.21.15.84:0/2195217266 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7ffb38059c90 con 0x7ffb40159130 2023-07-06T21:40:27.285 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:27.285 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:27.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb2406ace0 msgr2=0x7ffb2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb2406ace0 0x7ffb2406d190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009ab0 tx=0x7ffb34008040 comp rx=0 tx=0).stop 2023-07-06T21:40:27.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 msgr2=0x7ffb401f58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 0x7ffb401f58d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffb38014040 tx=0x7ffb38004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 shutdown_connections 2023-07-06T21:40:27.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb2406ace0 0x7ffb2406d190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb40159130 0x7ffb401f58d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb40070940 0x7ffb401f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 --2- 172.21.15.84:0/2195217266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb400c5000 0x7ffb4015a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 >> 172.21.15.84:0/2195217266 conn(0x7ffb400cf3d0 msgr2=0x7ffb400cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:27.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 shutdown_connections 2023-07-06T21:40:27.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.284+0000 7ffb46bff700 1 -- 172.21.15.84:0/2195217266 wait complete. 2023-07-06T21:40:27.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:27.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:27.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.497+0000 7ff9f75ff700 1 Processor -- start 2023-07-06T21:40:27.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.497+0000 7ff9f75ff700 1 -- start start 2023-07-06T21:40:27.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.498+0000 7ff9f75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00b81c0 0x7ff9f00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f75ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff9f00ceda0 0x7ff9f00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f75ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f00d1830 con 0x7ff9f00b1a80 2023-07-06T21:40:27.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f75ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9f00cf330 con 0x7ff9f00ceda0 2023-07-06T21:40:27.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f00d1270 con 0x7ff9f00b81c0 2023-07-06T21:40:27.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f4b9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff9f00b1a80 0x7ff9f015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46198/0 (socket says 172.21.15.84:46198) 2023-07-06T21:40:27.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f4b9a700 1 -- 172.21.15.84:0/162819735 learned_addr learned my addr 172.21.15.84:0/162819735 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:27.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f5b9c700 1 --2- 172.21.15.84:0/162819735 >> v2:172.21.15.84:3300/0 conn(0x7ff9f00ceda0 0x7ff9f00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f539b700 1 --2- 172.21.15.84:0/162819735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00b81c0 0x7ff9f00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9e7fff700 1 -- 172.21.15.84:0/162819735 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48514279 0 0) 0x7ff9f00d1830 con 0x7ff9f00b1a80 2023-07-06T21:40:27.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9e7fff700 1 -- 172.21.15.84:0/162819735 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9dc0035b0 con 0x7ff9f00b1a80 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f5b9c700 1 -- 172.21.15.84:0/162819735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00b81c0 msgr2=0x7ff9f00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f5b9c700 1 --2- 172.21.15.84:0/162819735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00b81c0 0x7ff9f00cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f5b9c700 1 -- 172.21.15.84:0/162819735 >> v1:172.21.15.84:6789/0 conn(0x7ff9f00b1a80 legacy=0x7ff9f015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.499+0000 7ff9f5b9c700 1 -- 172.21.15.84:0/162819735 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00d1550 con 0x7ff9f00ceda0 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.500+0000 7ff9f5b9c700 1 --2- 172.21.15.84:0/162819735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00ceda0 0x7ff9f00cfb40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009d90 tx=0x7ff9e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb51ae2171c719f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.500+0000 7ff9e7fff700 1 -- 172.21.15.84:0/162819735 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e800e070 con 0x7ff9f00ceda0 2023-07-06T21:40:27.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.500+0000 7ff9e7fff700 1 -- 172.21.15.84:0/162819735 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800bb20 con 0x7ff9f00ceda0 2023-07-06T21:40:27.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.500+0000 7ff9e7fff700 1 -- 172.21.15.84:0/162819735 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8012430 con 0x7ff9f00ceda0 2023-07-06T21:40:27.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.500+0000 7ff9f75ff700 1 -- 172.21.15.84:0/162819735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00ceda0 msgr2=0x7ff9f00cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.500+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/162819735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00ceda0 0x7ff9f00cfb40 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009d90 tx=0x7ff9e800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:27.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.501+0000 7ff9f75ff700 1 -- 172.21.15.84:0/162819735 shutdown_connections 2023-07-06T21:40:27.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.501+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/162819735 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00ceda0 0x7ff9f00cfb40 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.501+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/162819735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00b81c0 0x7ff9f00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.501+0000 7ff9f75ff700 1 -- 172.21.15.84:0/162819735 >> 172.21.15.84:0/162819735 conn(0x7ff9f006b2b0 msgr2=0x7ff9f006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:27.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.501+0000 7ff9f75ff700 1 -- 172.21.15.84:0/162819735 shutdown_connections 2023-07-06T21:40:27.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.501+0000 7ff9f75ff700 1 -- 172.21.15.84:0/162819735 wait complete. 2023-07-06T21:40:27.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.502+0000 7ff9f75ff700 1 Processor -- start 2023-07-06T21:40:27.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.502+0000 7ff9f75ff700 1 -- start start 2023-07-06T21:40:27.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.503+0000 7ff9f75ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 0x7ff9f01e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.503+0000 7ff9f75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9f00b81c0 0x7ff9f00cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.503+0000 7ff9f75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00ceda0 0x7ff9f00c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.503+0000 7ff9f75ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0148400 con 0x7ff9f00b1a80 2023-07-06T21:40:27.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.503+0000 7ff9f75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f01477f0 con 0x7ff9f00b81c0 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.503+0000 7ff9f75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f0147ad0 con 0x7ff9f00ceda0 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 0x7ff9f01e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 0x7ff9f01e1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45998/0 (socket says 172.21.15.84:45998) 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 -- 172.21.15.84:0/3893762135 learned_addr learned my addr 172.21.15.84:0/3893762135 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f5b9c700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00ceda0 0x7ff9f00c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f4b9a700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9f00b81c0 0x7ff9f00cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 -- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00ceda0 msgr2=0x7ff9f00c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00ceda0 0x7ff9f00c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 -- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9f00b81c0 msgr2=0x7ff9f00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9f00b81c0 0x7ff9f00cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 -- 172.21.15.84:0/3893762135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e8009a70 con 0x7ff9f00b1a80 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9f539b700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 0x7ff9f01e1730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00e9b0 tx=0x7ff9ec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.504+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec009710 con 0x7ff9f00b1a80 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.505+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ec00c850 con 0x7ff9f00b1a80 2023-07-06T21:40:27.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.505+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ec00cbc0 con 0x7ff9f00b1a80 2023-07-06T21:40:27.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.505+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f0147db0 con 0x7ff9f00b1a80 2023-07-06T21:40:27.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.505+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f00c6020 con 0x7ff9f00b1a80 2023-07-06T21:40:27.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.506+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f004e1d0 con 0x7ff9f00b1a80 2023-07-06T21:40:27.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.507+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff9ec00cd20 con 0x7ff9f00b1a80 2023-07-06T21:40:27.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.507+0000 7ff9e5ffb700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff9d406ace0 0x7ff9d406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.508+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff9ec089540 con 0x7ff9f00b1a80 2023-07-06T21:40:27.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.508+0000 7ff9f4b9a700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff9d406ace0 0x7ff9d406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.508+0000 7ff9f4b9a700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff9d406ace0 0x7ff9d406d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00cc240 tx=0x7ff9e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.515 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.512+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff9ec0580d0 con 0x7ff9f00b1a80 2023-07-06T21:40:27.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.707+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff9f00b4f30 con 0x7ff9f00b1a80 2023-07-06T21:40:27.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.708+0000 7ff9e5ffb700 1 -- 172.21.15.84:0/3893762135 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9ec057c60 con 0x7ff9f00b1a80 2023-07-06T21:40:27.711 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:27.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff9d406ace0 msgr2=0x7ff9d406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff9d406ace0 0x7ff9d406d190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00cc240 tx=0x7ff9e0009040 comp rx=0 tx=0).stop 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 msgr2=0x7ff9f01e1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 0x7ff9f01e1730 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec00e9b0 tx=0x7ff9ec00ed70 comp rx=0 tx=0).stop 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 shutdown_connections 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff9d406ace0 0x7ff9d406d190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff9f00ceda0 0x7ff9f00c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9f00b1a80 0x7ff9f01e1730 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 --2- 172.21.15.84:0/3893762135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff9f00b81c0 0x7ff9f00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 >> 172.21.15.84:0/3893762135 conn(0x7ff9f006b2b0 msgr2=0x7ff9f006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 shutdown_connections 2023-07-06T21:40:27.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.710+0000 7ff9f75ff700 1 -- 172.21.15.84:0/3893762135 wait complete. 2023-07-06T21:40:27.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:27.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.924+0000 7f3b951ff700 1 Processor -- start 2023-07-06T21:40:27.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.924+0000 7f3b951ff700 1 -- start start 2023-07-06T21:40:27.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b951ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b900b81c0 0x7f3b900cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b951ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3b900b1a80 0x7f3b900cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b951ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b900d1830 con 0x7f3b900ceda0 2023-07-06T21:40:27.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b951ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b900cf330 con 0x7f3b900b81c0 2023-07-06T21:40:27.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b951ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b900d1270 con 0x7f3b900b1a80 2023-07-06T21:40:27.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b8ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b900b81c0 0x7f3b900cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b8ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b900b81c0 0x7f3b900cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46010/0 (socket says 172.21.15.84:46010) 2023-07-06T21:40:27.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/978851683 learned_addr learned my addr 172.21.15.84:0/978851683 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:27.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.925+0000 7f3b8e59c700 1 --2- 172.21.15.84:0/978851683 >> v2:172.21.15.107:3300/0 conn(0x7f3b900b1a80 0x7f3b900cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/978851683 >> v2:172.21.15.107:3300/0 conn(0x7f3b900b1a80 msgr2=0x7f3b900cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:27.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8ed9d700 1 --2- 172.21.15.84:0/978851683 >> v2:172.21.15.107:3300/0 conn(0x7f3b900b1a80 0x7f3b900cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/978851683 >> v1:172.21.15.107:6789/0 conn(0x7f3b900ceda0 legacy=0x7f3b9015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/978851683 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b900d1550 con 0x7f3b900b81c0 2023-07-06T21:40:27.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8ed9d700 1 --2- 172.21.15.84:0/978851683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b81c0 0x7f3b900cf600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b900d0770 tx=0x7f3b80009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7a584aea8c4798 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8dd9b700 1 -- 172.21.15.84:0/978851683 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80011830 con 0x7f3b900b81c0 2023-07-06T21:40:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8dd9b700 1 -- 172.21.15.84:0/978851683 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b80011990 con 0x7f3b900b81c0 2023-07-06T21:40:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.926+0000 7f3b8dd9b700 1 -- 172.21.15.84:0/978851683 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80011d20 con 0x7f3b900b81c0 2023-07-06T21:40:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 -- 172.21.15.84:0/978851683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b81c0 msgr2=0x7f3b900cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 --2- 172.21.15.84:0/978851683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b81c0 0x7f3b900cf600 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3b900d0770 tx=0x7f3b80009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 -- 172.21.15.84:0/978851683 shutdown_connections 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 --2- 172.21.15.84:0/978851683 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b81c0 0x7f3b900cf600 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 --2- 172.21.15.84:0/978851683 >> v2:172.21.15.107:3300/0 conn(0x7f3b900b1a80 0x7f3b900cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 -- 172.21.15.84:0/978851683 >> 172.21.15.84:0/978851683 conn(0x7f3b9006b2b0 msgr2=0x7f3b9006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 -- 172.21.15.84:0/978851683 shutdown_connections 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.927+0000 7f3b951ff700 1 -- 172.21.15.84:0/978851683 wait complete. 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.929+0000 7f3b951ff700 1 Processor -- start 2023-07-06T21:40:27.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.929+0000 7f3b951ff700 1 -- start start 2023-07-06T21:40:27.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b951ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 0x7f3b900c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b951ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b900b81c0 0x7f3b9014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b951ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b900ceda0 0x7f3b9014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b951ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900c8260 con 0x7f3b900b1a80 2023-07-06T21:40:27.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b951ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900c7650 con 0x7f3b900b81c0 2023-07-06T21:40:27.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b951ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900c7930 con 0x7f3b900ceda0 2023-07-06T21:40:27.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 0x7f3b900c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.940 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 0x7f3b900c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46016/0 (socket says 172.21.15.84:46016) 2023-07-06T21:40:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/133548410 learned_addr learned my addr 172.21.15.84:0/133548410 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b900ceda0 msgr2=0x7f3b9014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8e59c700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b900b81c0 0x7f3b9014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8f59e700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b900ceda0 0x7f3b9014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b900ceda0 0x7f3b9014e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b900b81c0 msgr2=0x7f3b9014dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b900b81c0 0x7f3b9014dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8ed9d700 1 -- 172.21.15.84:0/133548410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80009550 con 0x7f3b900b1a80 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8e59c700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b900b81c0 0x7f3b9014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.930+0000 7f3b8f59e700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b900ceda0 0x7f3b9014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.931+0000 7f3b8ed9d700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 0x7f3b900c8ca0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3b80000c00 tx=0x7f3b80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.931+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b80004570 con 0x7f3b900b1a80 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.931+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8001f070 con 0x7f3b900b1a80 2023-07-06T21:40:27.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.931+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b900c7c10 con 0x7f3b900b1a80 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.931+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800038e0 con 0x7f3b900b1a80 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.931+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b900c91e0 con 0x7f3b900b1a80 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.932+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b900c9370 con 0x7f3b900b1a80 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.933+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b80021030 con 0x7f3b900b1a80 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.934+0000 7f3b777fe700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b7c06ace0 0x7f3b7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.934+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3b8008a9b0 con 0x7f3b900b1a80 2023-07-06T21:40:27.943 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.934+0000 7f3b8e59c700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b7c06ace0 0x7f3b7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:27.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.934+0000 7f3b8e59c700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b7c06ace0 0x7f3b7c06d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b900ccd20 tx=0x7f3b84008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:27.944 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:27.938+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b80055a10 con 0x7f3b900b1a80 2023-07-06T21:40:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.133+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3b9004e1d0 con 0x7f3b900b1a80 2023-07-06T21:40:28.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.133+0000 7f3b777fe700 1 -- 172.21.15.84:0/133548410 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3b800150a0 con 0x7f3b900b1a80 2023-07-06T21:40:28.137 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b7c06ace0 msgr2=0x7f3b7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b7c06ace0 0x7f3b7c06d190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3b900ccd20 tx=0x7f3b84008040 comp rx=0 tx=0).stop 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 msgr2=0x7f3b900c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 0x7f3b900c8ca0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3b80000c00 tx=0x7f3b80004750 comp rx=0 tx=0).stop 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 shutdown_connections 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b7c06ace0 0x7f3b7c06d190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b900ceda0 0x7f3b9014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b900b1a80 0x7f3b900c8ca0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 --2- 172.21.15.84:0/133548410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b900b81c0 0x7f3b9014dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.135+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 >> 172.21.15.84:0/133548410 conn(0x7f3b9006b2b0 msgr2=0x7f3b900cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.136+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 shutdown_connections 2023-07-06T21:40:28.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.136+0000 7f3b951ff700 1 -- 172.21.15.84:0/133548410 wait complete. 2023-07-06T21:40:28.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:28.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.348+0000 7fb36c1ff700 1 Processor -- start 2023-07-06T21:40:28.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.348+0000 7fb36c1ff700 1 -- start start 2023-07-06T21:40:28.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.349+0000 7fb36c1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb3640cfd10 0x7fb3640d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.349+0000 7fb36c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3640d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.349+0000 7fb36c1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640d1830 con 0x7fb3640b81c0 2023-07-06T21:40:28.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.349+0000 7fb36c1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3640cf330 con 0x7fb3640cfd10 2023-07-06T21:40:28.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.349+0000 7fb36c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640d1270 con 0x7fb3640b1a80 2023-07-06T21:40:28.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb369f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb3640b81c0 0x7fb3640cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46210/0 (socket says 172.21.15.84:46210) 2023-07-06T21:40:28.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb369f9b700 1 -- 172.21.15.84:0/1462705588 learned_addr learned my addr 172.21.15.84:0/1462705588 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:28.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb36a79c700 1 --2- 172.21.15.84:0/1462705588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3640d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb36979a700 1 --2- 172.21.15.84:0/1462705588 >> v2:172.21.15.107:3300/0 conn(0x7fb3640cfd10 0x7fb3640d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb368f99700 1 -- 172.21.15.84:0/1462705588 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756313013 0 0) 0x7fb3640d1830 con 0x7fb3640b81c0 2023-07-06T21:40:28.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb368f99700 1 -- 172.21.15.84:0/1462705588 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3500035b0 con 0x7fb3640b81c0 2023-07-06T21:40:28.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb36a79c700 1 -- 172.21.15.84:0/1462705588 >> v2:172.21.15.107:3300/0 conn(0x7fb3640cfd10 msgr2=0x7fb3640d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:28.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb36a79c700 1 --2- 172.21.15.84:0/1462705588 >> v2:172.21.15.107:3300/0 conn(0x7fb3640cfd10 0x7fb3640d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb36a79c700 1 -- 172.21.15.84:0/1462705588 >> v1:172.21.15.84:6789/0 conn(0x7fb3640b81c0 legacy=0x7fb3640cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb368f99700 1 -- 172.21.15.84:0/1462705588 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1106700910 0 0) 0x7fb3500035b0 con 0x7fb3640b81c0 2023-07-06T21:40:28.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.350+0000 7fb36a79c700 1 -- 172.21.15.84:0/1462705588 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640d1550 con 0x7fb3640b1a80 2023-07-06T21:40:28.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.351+0000 7fb36a79c700 1 --2- 172.21.15.84:0/1462705588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3640d28c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009d90 tx=0x7fb35c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e10a8a617200f40 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:28.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.351+0000 7fb368f99700 1 -- 172.21.15.84:0/1462705588 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c00e070 con 0x7fb3640b1a80 2023-07-06T21:40:28.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.351+0000 7fb368f99700 1 -- 172.21.15.84:0/1462705588 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c003a20 con 0x7fb3640b1a80 2023-07-06T21:40:28.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.351+0000 7fb368f99700 1 -- 172.21.15.84:0/1462705588 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360001670 con 0x7fb3640b1a80 2023-07-06T21:40:28.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.351+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1462705588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 msgr2=0x7fb3640d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.351+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1462705588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3640d28c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009d90 tx=0x7fb35c00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:28.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.352+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1462705588 shutdown_connections 2023-07-06T21:40:28.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.352+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1462705588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3640d28c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.352+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1462705588 >> v2:172.21.15.107:3300/0 conn(0x7fb3640cfd10 0x7fb3640d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.352+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1462705588 >> 172.21.15.84:0/1462705588 conn(0x7fb36406b2b0 msgr2=0x7fb36406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:28.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.352+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1462705588 shutdown_connections 2023-07-06T21:40:28.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.352+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1462705588 wait complete. 2023-07-06T21:40:28.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.353+0000 7fb36c1ff700 1 Processor -- start 2023-07-06T21:40:28.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.353+0000 7fb36c1ff700 1 -- start start 2023-07-06T21:40:28.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.354+0000 7fb36c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3641f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.354+0000 7fb36c1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3640b81c0 0x7fb36415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.354+0000 7fb36c1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 0x7fb36414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.354+0000 7fb36c1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415e310 con 0x7fb3640cfd10 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.354+0000 7fb36c1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415d270 con 0x7fb3640b81c0 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.354+0000 7fb36c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415d550 con 0x7fb3640b1a80 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 0x7fb36414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 0x7fb36414df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46018/0 (socket says 172.21.15.84:46018) 2023-07-06T21:40:28.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 -- 172.21.15.84:0/1210286563 learned_addr learned my addr 172.21.15.84:0/1210286563 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:28.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36979a700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3640b81c0 0x7fb36415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb369f9b700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3641f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 -- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 msgr2=0x7fb3641f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3641f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 -- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3640b81c0 msgr2=0x7fb36415ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3640b81c0 0x7fb36415ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 -- 172.21.15.84:0/1210286563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c009a70 con 0x7fb3640cfd10 2023-07-06T21:40:28.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb369f9b700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3641f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36a79c700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 0x7fb36414df40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb35c004000 tx=0x7fb35c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0044c0 con 0x7fb3640cfd10 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c004640 con 0x7fb3640cfd10 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c012430 con 0x7fb3640cfd10 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.355+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36415d830 con 0x7fb3640cfd10 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.356+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36415ddf0 con 0x7fb3640cfd10 2023-07-06T21:40:28.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.357+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36404e1d0 con 0x7fb3640cfd10 2023-07-06T21:40:28.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.357+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb35c012590 con 0x7fb3640cfd10 2023-07-06T21:40:28.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.358+0000 7fb35a7fc700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb34806ace0 0x7fb34806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.358+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb35c08b0c0 con 0x7fb3640cfd10 2023-07-06T21:40:28.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.359+0000 7fb369f9b700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb34806ace0 0x7fb34806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.359+0000 7fb369f9b700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb34806ace0 0x7fb34806d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb360001eb0 tx=0x7fb360001d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:28.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.363+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb35c056120 con 0x7fb3640cfd10 2023-07-06T21:40:28.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.558+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb3640b4f30 con 0x7fb3640cfd10 2023-07-06T21:40:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.558+0000 7fb35a7fc700 1 -- 172.21.15.84:0/1210286563 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb35c059740 con 0x7fb3640cfd10 2023-07-06T21:40:28.562 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.560+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb34806ace0 msgr2=0x7fb34806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.560+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb34806ace0 0x7fb34806d190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb360001eb0 tx=0x7fb360001d80 comp rx=0 tx=0).stop 2023-07-06T21:40:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.560+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 msgr2=0x7fb36414df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.560+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 0x7fb36414df40 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb35c004000 tx=0x7fb35c004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 shutdown_connections 2023-07-06T21:40:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb34806ace0 0x7fb34806d190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3640cfd10 0x7fb36414df40 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb3640b1a80 0x7fb3641f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 --2- 172.21.15.84:0/1210286563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3640b81c0 0x7fb36415ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 >> 172.21.15.84:0/1210286563 conn(0x7fb36406b2b0 msgr2=0x7fb36415bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 shutdown_connections 2023-07-06T21:40:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.561+0000 7fb36c1ff700 1 -- 172.21.15.84:0/1210286563 wait complete. 2023-07-06T21:40:28.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:28.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.774+0000 7fa493dff700 1 Processor -- start 2023-07-06T21:40:28.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.775+0000 7fa493dff700 1 -- start start 2023-07-06T21:40:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa493dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa48c0b81c0 0x7fa48c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa493dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa48c0b1a80 0x7fa48c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa493dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48c0d1830 con 0x7fa48c0ceda0 2023-07-06T21:40:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa493dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c0cf330 con 0x7fa48c0b81c0 2023-07-06T21:40:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa493dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa48c0d1270 con 0x7fa48c0b1a80 2023-07-06T21:40:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa491b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa48c0b81c0 0x7fa48c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa491b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa48c0b81c0 0x7fa48c0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46028/0 (socket says 172.21.15.84:46028) 2023-07-06T21:40:28.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa491b9b700 1 -- 172.21.15.84:0/583460883 learned_addr learned my addr 172.21.15.84:0/583460883 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:28.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.776+0000 7fa49139a700 1 --2- 172.21.15.84:0/583460883 >> v2:172.21.15.107:3300/0 conn(0x7fa48c0b1a80 0x7fa48c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa491b9b700 1 -- 172.21.15.84:0/583460883 >> v2:172.21.15.107:3300/0 conn(0x7fa48c0b1a80 msgr2=0x7fa48c0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa491b9b700 1 --2- 172.21.15.84:0/583460883 >> v2:172.21.15.107:3300/0 conn(0x7fa48c0b1a80 0x7fa48c0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa491b9b700 1 -- 172.21.15.84:0/583460883 >> v1:172.21.15.107:6789/0 conn(0x7fa48c0ceda0 legacy=0x7fa48c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa491b9b700 1 -- 172.21.15.84:0/583460883 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c0d1550 con 0x7fa48c0b81c0 2023-07-06T21:40:28.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa491b9b700 1 --2- 172.21.15.84:0/583460883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b81c0 0x7fa48c0cf600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0d0770 tx=0x7fa488009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c600dd95adaac21 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:28.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa490b99700 1 -- 172.21.15.84:0/583460883 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488011830 con 0x7fa48c0b81c0 2023-07-06T21:40:28.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa490b99700 1 -- 172.21.15.84:0/583460883 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa488011990 con 0x7fa48c0b81c0 2023-07-06T21:40:28.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.777+0000 7fa490b99700 1 -- 172.21.15.84:0/583460883 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488011d20 con 0x7fa48c0b81c0 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 -- 172.21.15.84:0/583460883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b81c0 msgr2=0x7fa48c0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 --2- 172.21.15.84:0/583460883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b81c0 0x7fa48c0cf600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0d0770 tx=0x7fa488009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 -- 172.21.15.84:0/583460883 shutdown_connections 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 --2- 172.21.15.84:0/583460883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b81c0 0x7fa48c0cf600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 --2- 172.21.15.84:0/583460883 >> v2:172.21.15.107:3300/0 conn(0x7fa48c0b1a80 0x7fa48c0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 -- 172.21.15.84:0/583460883 >> 172.21.15.84:0/583460883 conn(0x7fa48c06b2b0 msgr2=0x7fa48c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:28.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 -- 172.21.15.84:0/583460883 shutdown_connections 2023-07-06T21:40:28.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.778+0000 7fa493dff700 1 -- 172.21.15.84:0/583460883 wait complete. 2023-07-06T21:40:28.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.779+0000 7fa493dff700 1 Processor -- start 2023-07-06T21:40:28.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.780+0000 7fa493dff700 1 -- start start 2023-07-06T21:40:28.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.780+0000 7fa493dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 0x7fa48c155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.780+0000 7fa493dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa48c0b81c0 0x7fa48c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.780+0000 7fa493dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa48c0ceda0 0x7fa48c15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa493dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0c6a90 con 0x7fa48c0b1a80 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa493dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0c6d70 con 0x7fa48c0ceda0 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa493dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c0c7050 con 0x7fa48c0b81c0 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 0x7fa48c155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 0x7fa48c155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46034/0 (socket says 172.21.15.84:46034) 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 -- 172.21.15.84:0/2048230515 learned_addr learned my addr 172.21.15.84:0/2048230515 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:28.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa49239c700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa48c0ceda0 0x7fa48c15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa49139a700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa48c0b81c0 0x7fa48c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 -- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa48c0b81c0 msgr2=0x7fa48c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa48c0b81c0 0x7fa48c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 -- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa48c0ceda0 msgr2=0x7fa48c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa48c0ceda0 0x7fa48c15a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 -- 172.21.15.84:0/2048230515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa488009550 con 0x7fa48c0b1a80 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa49239c700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa48c0ceda0 0x7fa48c15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.781+0000 7fa491b9b700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 0x7fa48c155ce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa488000c00 tx=0x7fa488004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.782+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488004570 con 0x7fa48c0b1a80 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.782+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48801f070 con 0x7fa48c0b1a80 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.782+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c0c7330 con 0x7fa48c0b1a80 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.782+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4880038e0 con 0x7fa48c0b1a80 2023-07-06T21:40:28.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.782+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c156220 con 0x7fa48c0b1a80 2023-07-06T21:40:28.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.783+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c0c7850 con 0x7fa48c0b1a80 2023-07-06T21:40:28.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.784+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa488021030 con 0x7fa48c0b1a80 2023-07-06T21:40:28.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.785+0000 7fa4827fc700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa47006ace0 0x7fa47006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:28.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.785+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa48808a9b0 con 0x7fa48c0b1a80 2023-07-06T21:40:28.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.785+0000 7fa49139a700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa47006ace0 0x7fa47006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:28.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.786+0000 7fa49139a700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa47006ace0 0x7fa47006d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0c6470 tx=0x7fa47c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:28.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.789+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa48c0c7850 con 0x7fa48c0b1a80 2023-07-06T21:40:28.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.987+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa48c0c7850 con 0x7fa48c0b1a80 2023-07-06T21:40:28.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.988+0000 7fa4827fc700 1 -- 172.21.15.84:0/2048230515 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa48c0c7850 con 0x7fa48c0b1a80 2023-07-06T21:40:28.992 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:28.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa47006ace0 msgr2=0x7fa47006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa47006ace0 0x7fa47006d190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0c6470 tx=0x7fa47c008040 comp rx=0 tx=0).stop 2023-07-06T21:40:28.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 msgr2=0x7fa48c155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:28.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 0x7fa48c155ce0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa488000c00 tx=0x7fa488004750 comp rx=0 tx=0).stop 2023-07-06T21:40:28.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 shutdown_connections 2023-07-06T21:40:28.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa47006ace0 0x7fa47006d190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa48c0ceda0 0x7fa48c15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa48c0b1a80 0x7fa48c155ce0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 --2- 172.21.15.84:0/2048230515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa48c0b81c0 0x7fa48c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:28.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.990+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 >> 172.21.15.84:0/2048230515 conn(0x7fa48c06b2b0 msgr2=0x7fa48c0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:28.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.991+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 shutdown_connections 2023-07-06T21:40:28.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:28.991+0000 7fa493dff700 1 -- 172.21.15.84:0/2048230515 wait complete. 2023-07-06T21:40:29.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:29.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.202+0000 7f5f42fff700 1 Processor -- start 2023-07-06T21:40:29.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.203+0000 7f5f42fff700 1 -- start start 2023-07-06T21:40:29.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f42fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5f3c0b83c0 0x7f5f3c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f42fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5f3c0cfc60 0x7f5f3c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f42fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3c0d0a80 con 0x7f5f3c0d3710 2023-07-06T21:40:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f42fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0b8790 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f42fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3c0b4e70 con 0x7f5f3c0cfc60 2023-07-06T21:40:29.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f3bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5f3c0d3710 0x7f5f3c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46212/0 (socket says 172.21.15.84:46212) 2023-07-06T21:40:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f3bfff700 1 -- 172.21.15.84:0/2133435240 learned_addr learned my addr 172.21.15.84:0/2133435240 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.204+0000 7f5f40d9b700 1 --2- 172.21.15.84:0/2133435240 >> v2:172.21.15.84:3300/0 conn(0x7f5f3c0b83c0 0x7f5f3c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f40d9b700 1 -- 172.21.15.84:0/2133435240 >> v2:172.21.15.107:3300/0 conn(0x7f5f3c0cfc60 msgr2=0x7f5f3c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:29.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f40d9b700 1 --2- 172.21.15.84:0/2133435240 >> v2:172.21.15.107:3300/0 conn(0x7f5f3c0cfc60 0x7f5f3c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f40d9b700 1 -- 172.21.15.84:0/2133435240 >> v1:172.21.15.84:6789/0 conn(0x7f5f3c0d3710 legacy=0x7f5f3c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f40d9b700 1 -- 172.21.15.84:0/2133435240 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c0b5150 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f40d9b700 1 --2- 172.21.15.84:0/2133435240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c0cd200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0d0030 tx=0x7f5f30009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3be0e97968459f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:29.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f3b7fe700 1 -- 172.21.15.84:0/2133435240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011830 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f3b7fe700 1 -- 172.21.15.84:0/2133435240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f30011990 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f3b7fe700 1 -- 172.21.15.84:0/2133435240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011d20 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f42fff700 1 -- 172.21.15.84:0/2133435240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 msgr2=0x7f5f3c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.205+0000 7f5f42fff700 1 --2- 172.21.15.84:0/2133435240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c0cd200 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0d0030 tx=0x7f5f30009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.206+0000 7f5f42fff700 1 -- 172.21.15.84:0/2133435240 shutdown_connections 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.206+0000 7f5f42fff700 1 --2- 172.21.15.84:0/2133435240 >> v2:172.21.15.107:3300/0 conn(0x7f5f3c0cfc60 0x7f5f3c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.206+0000 7f5f42fff700 1 --2- 172.21.15.84:0/2133435240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c0cd200 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.206+0000 7f5f42fff700 1 -- 172.21.15.84:0/2133435240 >> 172.21.15.84:0/2133435240 conn(0x7f5f3c06b350 msgr2=0x7f5f3c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.206+0000 7f5f42fff700 1 -- 172.21.15.84:0/2133435240 shutdown_connections 2023-07-06T21:40:29.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.206+0000 7f5f42fff700 1 -- 172.21.15.84:0/2133435240 wait complete. 2023-07-06T21:40:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.208+0000 7f5f42fff700 1 Processor -- start 2023-07-06T21:40:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.208+0000 7f5f42fff700 1 -- start start 2023-07-06T21:40:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.209+0000 7f5f42fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.209+0000 7f5f42fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f3c0cfc60 0x7f5f3c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.209+0000 7f5f42fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c0d3710 0x7f5f3c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.209+0000 7f5f42fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c155020 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.209+0000 7f5f42fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c151120 con 0x7f5f3c0d3710 2023-07-06T21:40:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.209+0000 7f5f42fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3c155270 con 0x7f5f3c0cfc60 2023-07-06T21:40:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46054/0 (socket says 172.21.15.84:46054) 2023-07-06T21:40:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 -- 172.21.15.84:0/3464322308 learned_addr learned my addr 172.21.15.84:0/3464322308 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 -- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f3c0cfc60 msgr2=0x7f5f3c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f3c0cfc60 0x7f5f3c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 -- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c0d3710 msgr2=0x7f5f3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f4159c700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c0d3710 0x7f5f3c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c0d3710 0x7f5f3c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 -- 172.21.15.84:0/3464322308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f300094a0 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f4159c700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c0d3710 0x7f5f3c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.210+0000 7f5f40d9b700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c150be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5f300090b0 tx=0x7f5f30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.211+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f300038a0 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.211+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3001e070 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.211+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3c155550 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.211+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30003c40 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.211+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3c155b10 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.212+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f3c15a5b0 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.213+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5f3002d450 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.214+0000 7f5f397fa700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5f2006ace0 0x7f5f2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.214+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5f3008a620 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.214+0000 7f5f3bfff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5f2006ace0 0x7f5f2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.215+0000 7f5f3bfff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5f2006ace0 0x7f5f2006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0d3b20 tx=0x7f5f28007ad0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:29.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.218+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5f30055720 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.431+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f3c0b24d0 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.431+0000 7f5f397fa700 1 -- 172.21.15.84:0/3464322308 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f30058d40 con 0x7f5f3c0b83c0 2023-07-06T21:40:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5f2006ace0 msgr2=0x7f5f2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5f2006ace0 0x7f5f2006d190 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0d3b20 tx=0x7f5f28007ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 msgr2=0x7f5f3c150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c150be0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5f300090b0 tx=0x7f5f30004750 comp rx=0 tx=0).stop 2023-07-06T21:40:29.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 shutdown_connections 2023-07-06T21:40:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5f2006ace0 0x7f5f2006d190 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f3c0d3710 0x7f5f3c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f3c0b83c0 0x7f5f3c150be0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 --2- 172.21.15.84:0/3464322308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f3c0cfc60 0x7f5f3c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 >> 172.21.15.84:0/3464322308 conn(0x7f5f3c06b350 msgr2=0x7f5f3c0b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:29.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 shutdown_connections 2023-07-06T21:40:29.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.433+0000 7f5f42fff700 1 -- 172.21.15.84:0/3464322308 wait complete. 2023-07-06T21:40:29.439 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:29.454 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:40:29.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:29.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-07-06T21:40:29.524 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:29.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:40:29.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:29.526 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T21:40:29.569 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:40:29.569 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:29.617 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:40:29.618 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:29.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.892+0000 7f93c07ff700 1 Processor -- start 2023-07-06T21:40:29.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.892+0000 7f93c07ff700 1 -- start start 2023-07-06T21:40:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93c07ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93b8158830 0x7f93b80c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93c07ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b80c5280 0x7f93b8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93c07ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b80c3f80 con 0x7f93b806a3d0 2023-07-06T21:40:29.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93c07ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b806a850 con 0x7f93b8158830 2023-07-06T21:40:29.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93c07ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80c4260 con 0x7f93b80c5280 2023-07-06T21:40:29.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93be59b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93b8158830 0x7f93b80c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93be59b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93b8158830 0x7f93b80c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46066/0 (socket says 172.21.15.84:46066) 2023-07-06T21:40:29.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93be59b700 1 -- 172.21.15.84:0/3092897756 learned_addr learned my addr 172.21.15.84:0/3092897756 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93be59b700 1 -- 172.21.15.84:0/3092897756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b80c5280 msgr2=0x7f93b8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:29.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.894+0000 7f93be59b700 1 --2- 172.21.15.84:0/3092897756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b80c5280 0x7f93b8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.895+0000 7f93be59b700 1 -- 172.21.15.84:0/3092897756 >> v1:172.21.15.107:6789/0 conn(0x7f93b806a3d0 legacy=0x7f93b80c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:29.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.895+0000 7f93be59b700 1 -- 172.21.15.84:0/3092897756 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80c4540 con 0x7f93b8158830 2023-07-06T21:40:29.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.895+0000 7f93be59b700 1 --2- 172.21.15.84:0/3092897756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b8158830 0x7f93b80c4d40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f93b40098f0 tx=0x7f93b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf368616cc63ecb0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.895+0000 7f93bd599700 1 -- 172.21.15.84:0/3092897756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4004230 con 0x7f93b8158830 2023-07-06T21:40:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.895+0000 7f93bd599700 1 -- 172.21.15.84:0/3092897756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4004390 con 0x7f93b8158830 2023-07-06T21:40:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.895+0000 7f93bd599700 1 -- 172.21.15.84:0/3092897756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40036a0 con 0x7f93b8158830 2023-07-06T21:40:29.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 -- 172.21.15.84:0/3092897756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b8158830 msgr2=0x7f93b80c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 --2- 172.21.15.84:0/3092897756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b8158830 0x7f93b80c4d40 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f93b40098f0 tx=0x7f93b4009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:29.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 -- 172.21.15.84:0/3092897756 shutdown_connections 2023-07-06T21:40:29.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 --2- 172.21.15.84:0/3092897756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b8158830 0x7f93b80c4d40 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 --2- 172.21.15.84:0/3092897756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b80c5280 0x7f93b8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 -- 172.21.15.84:0/3092897756 >> 172.21.15.84:0/3092897756 conn(0x7f93b80cf3d0 msgr2=0x7f93b80cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:29.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 -- 172.21.15.84:0/3092897756 shutdown_connections 2023-07-06T21:40:29.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.896+0000 7f93c07ff700 1 -- 172.21.15.84:0/3092897756 wait complete. 2023-07-06T21:40:29.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.897+0000 7f93c07ff700 1 Processor -- start 2023-07-06T21:40:29.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.898+0000 7f93c07ff700 1 -- start start 2023-07-06T21:40:29.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.898+0000 7f93c07ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b806a3d0 0x7f93b8157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.898+0000 7f93c07ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 0x7f93b81e4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93c07ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8158830 0x7f93b81477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93c07ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80b0b60 con 0x7f93b80c5280 2023-07-06T21:40:29.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93c07ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81e3bd0 con 0x7f93b8158830 2023-07-06T21:40:29.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93c07ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81e4b70 con 0x7f93b806a3d0 2023-07-06T21:40:29.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 0x7f93b81e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 0x7f93b81e4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46070/0 (socket says 172.21.15.84:46070) 2023-07-06T21:40:29.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 -- 172.21.15.84:0/369065043 learned_addr learned my addr 172.21.15.84:0/369065043 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:29.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bed9c700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8158830 0x7f93b81477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93be59b700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b806a3d0 0x7f93b8157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 -- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b806a3d0 msgr2=0x7f93b8157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b806a3d0 0x7f93b8157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 -- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8158830 msgr2=0x7f93b81477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8158830 0x7f93b81477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bdd9a700 1 -- 172.21.15.84:0/369065043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b4009550 con 0x7f93b80c5280 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93be59b700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b806a3d0 0x7f93b8157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.899+0000 7f93bed9c700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8158830 0x7f93b81477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:29.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.900+0000 7f93bdd9a700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 0x7f93b81e4630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b530 tx=0x7f93a800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.900+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8011830 con 0x7f93b80c5280 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.900+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a80119b0 con 0x7f93b80c5280 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.900+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b80ae580 con 0x7f93b80c5280 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.900+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8011d60 con 0x7f93b80c5280 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.900+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b80b1070 con 0x7f93b80c5280 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.901+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b80c39b0 con 0x7f93b80c5280 2023-07-06T21:40:29.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.902+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f93a801a400 con 0x7f93b80c5280 2023-07-06T21:40:29.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.903+0000 7f93aeffd700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f939c06ace0 0x7f939c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:29.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.903+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f93a8089640 con 0x7f93b80c5280 2023-07-06T21:40:29.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.903+0000 7f93be59b700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f939c06ace0 0x7f939c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:29.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.904+0000 7f93be59b700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f939c06ace0 0x7f939c06d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009c40 tx=0x7f93b4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:29.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:29.907+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f93a8054610 con 0x7f93b80c5280 2023-07-06T21:40:30.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.110+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93b80adac0 con 0x7f93b80c5280 2023-07-06T21:40:30.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.111+0000 7f93aeffd700 1 -- 172.21.15.84:0/369065043 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f93a8057c30 con 0x7f93b80c5280 2023-07-06T21:40:30.115 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:30.116 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f939c06ace0 msgr2=0x7f939c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f939c06ace0 0x7f939c06d190 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f93b4009c40 tx=0x7f93b4008030 comp rx=0 tx=0).stop 2023-07-06T21:40:30.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 msgr2=0x7f93b81e4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 0x7f93b81e4630 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b530 tx=0x7f93a800b8f0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 shutdown_connections 2023-07-06T21:40:30.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f939c06ace0 0x7f939c06d190 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93b8158830 0x7f93b81477b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93b806a3d0 0x7f93b8157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 --2- 172.21.15.84:0/369065043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93b80c5280 0x7f93b81e4630 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 >> 172.21.15.84:0/369065043 conn(0x7f93b80cf3d0 msgr2=0x7f93b80c7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 shutdown_connections 2023-07-06T21:40:30.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.114+0000 7f93c07ff700 1 -- 172.21.15.84:0/369065043 wait complete. 2023-07-06T21:40:30.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:30.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:30.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.336+0000 7f2f48bff700 1 Processor -- start 2023-07-06T21:40:30.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.336+0000 7f2f48bff700 1 -- start start 2023-07-06T21:40:30.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f48bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f440d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f48bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2f440b1a80 0x7f2f440d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f48bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f440d1830 con 0x7f2f440b81c0 2023-07-06T21:40:30.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f48bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f440cf330 con 0x7f2f440b1a80 2023-07-06T21:40:30.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f48bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440d1270 con 0x7f2f440cfd10 2023-07-06T21:40:30.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f42d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2f440b1a80 0x7f2f440d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f440d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f42d9d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2f440b1a80 0x7f2f440d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:36766/0 (socket says 172.21.15.84:36766) 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f4259c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2f440b81c0 0x7f2f440cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:43332/0 (socket says 172.21.15.84:43332) 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f440d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:48084/0 (socket says 172.21.15.84:48084) 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.338+0000 7f2f42d9d700 1 -- 172.21.15.84:0/2040042669 learned_addr learned my addr 172.21.15.84:0/2040042669 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f41d9b700 1 -- 172.21.15.84:0/2040042669 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492203279 0 0) 0x7f2f440d1830 con 0x7f2f440b81c0 2023-07-06T21:40:30.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f41d9b700 1 -- 172.21.15.84:0/2040042669 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f300035b0 con 0x7f2f440b81c0 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f42d9d700 1 -- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 msgr2=0x7f2f440d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f42d9d700 1 --2- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f440d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f42d9d700 1 -- 172.21.15.84:0/2040042669 >> v1:172.21.15.107:6789/0 conn(0x7f2f440b81c0 legacy=0x7f2f440cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f42d9d700 1 -- 172.21.15.84:0/2040042669 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440d1550 con 0x7f2f440b1a80 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f3bfff700 1 --2- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f440d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f42d9d700 1 --2- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440d28c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009e80 tx=0x7f2f3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a74118a6de26 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:30.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f41d9b700 1 -- 172.21.15.84:0/2040042669 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00e070 con 0x7f2f440b1a80 2023-07-06T21:40:30.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f41d9b700 1 -- 172.21.15.84:0/2040042669 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00b8d0 con 0x7f2f440b1a80 2023-07-06T21:40:30.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.339+0000 7f2f41d9b700 1 -- 172.21.15.84:0/2040042669 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00bc80 con 0x7f2f440b1a80 2023-07-06T21:40:30.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 -- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 msgr2=0x7f2f440d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 --2- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440d28c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009e80 tx=0x7f2f3c00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 -- 172.21.15.84:0/2040042669 shutdown_connections 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 --2- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440d28c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 --2- 172.21.15.84:0/2040042669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f440d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 -- 172.21.15.84:0/2040042669 >> 172.21.15.84:0/2040042669 conn(0x7f2f4406b2b0 msgr2=0x7f2f4406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 -- 172.21.15.84:0/2040042669 shutdown_connections 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.340+0000 7f2f48bff700 1 -- 172.21.15.84:0/2040042669 wait complete. 2023-07-06T21:40:30.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.342+0000 7f2f48bff700 1 Processor -- start 2023-07-06T21:40:30.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.342+0000 7f2f48bff700 1 -- start start 2023-07-06T21:40:30.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f48bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f48bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 0x7f2f4414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f48bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f441f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f48bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4415de10 con 0x7f2f440b81c0 2023-07-06T21:40:30.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f48bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4415dae0 con 0x7f2f440b1a80 2023-07-06T21:40:30.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f48bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441f2740 con 0x7f2f440cfd10 2023-07-06T21:40:30.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 0x7f2f4414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 0x7f2f4414fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46084/0 (socket says 172.21.15.84:46084) 2023-07-06T21:40:30.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.343+0000 7f2f3bfff700 1 -- 172.21.15.84:0/4154402815 learned_addr learned my addr 172.21.15.84:0/4154402815 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:30.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f42d9d700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f441f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f4259c700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3bfff700 1 -- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 msgr2=0x7f2f441f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3bfff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f441f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3bfff700 1 -- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 msgr2=0x7f2f440cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3bfff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440cc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3bfff700 1 -- 172.21.15.84:0/4154402815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c0094a0 con 0x7f2f440b81c0 2023-07-06T21:40:30.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3bfff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 0x7f2f4414fff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9b0 tx=0x7f2f2c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c009700 con 0x7f2f440b81c0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c00c850 con 0x7f2f440b81c0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c009b30 con 0x7f2f440b81c0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c00cc00 con 0x7f2f440b81c0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.344+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f441f2d00 con 0x7f2f440b81c0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.345+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4404e1d0 con 0x7f2f440b81c0 2023-07-06T21:40:30.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.346+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f2c013470 con 0x7f2f440b81c0 2023-07-06T21:40:30.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.348+0000 7f2f3affd700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f2406ace0 0x7f2f2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.348+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2f2c089300 con 0x7f2f440b81c0 2023-07-06T21:40:30.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.348+0000 7f2f4259c700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f2406ace0 0x7f2f2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.348+0000 7f2f4259c700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f2406ace0 0x7f2f2406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f340011e0 tx=0x7f2f34007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:30.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.352+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f2c057d60 con 0x7f2f440b81c0 2023-07-06T21:40:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.552+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f4415df50 con 0x7f2f440b81c0 2023-07-06T21:40:30.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.552+0000 7f2f3affd700 1 -- 172.21.15.84:0/4154402815 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f2c0578f0 con 0x7f2f440b81c0 2023-07-06T21:40:30.556 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:30.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.554+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f2406ace0 msgr2=0x7f2f2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.554+0000 7f2f48bff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f2406ace0 0x7f2f2406d190 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2f340011e0 tx=0x7f2f34007c00 comp rx=0 tx=0).stop 2023-07-06T21:40:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.554+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 msgr2=0x7f2f4414fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.554+0000 7f2f48bff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 0x7f2f4414fff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00e9b0 tx=0x7f2f2c00ecc0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 shutdown_connections 2023-07-06T21:40:30.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f2406ace0 0x7f2f2406d190 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f440cfd10 0x7f2f441f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f440b1a80 0x7f2f440cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 --2- 172.21.15.84:0/4154402815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f440b81c0 0x7f2f4414fff0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 >> 172.21.15.84:0/4154402815 conn(0x7f2f4406b2b0 msgr2=0x7f2f440b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 shutdown_connections 2023-07-06T21:40:30.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.555+0000 7f2f48bff700 1 -- 172.21.15.84:0/4154402815 wait complete. 2023-07-06T21:40:30.576 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.768+0000 7f43dc5ff700 1 Processor -- start 2023-07-06T21:40:30.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.768+0000 7f43dc5ff700 1 -- start start 2023-07-06T21:40:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43dc5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40b1a80 0x7f43d40d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43dc5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d40d1830 con 0x7f43d40cfd10 2023-07-06T21:40:30.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43dc5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d40cf330 con 0x7f43d40b81c0 2023-07-06T21:40:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43dc5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40d1270 con 0x7f43d40b1a80 2023-07-06T21:40:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43d9b9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f43d40cfd10 0x7f43d40d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46216/0 (socket says 172.21.15.84:46216) 2023-07-06T21:40:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43d9b9a700 1 -- 172.21.15.84:0/2969912969 learned_addr learned my addr 172.21.15.84:0/2969912969 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:30.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.770+0000 7f43dab9c700 1 --2- 172.21.15.84:0/2969912969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40b1a80 0x7f43d40d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661840148 0 0) 0x7f43d40d1830 con 0x7f43d40cfd10 2023-07-06T21:40:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43c00035b0 con 0x7f43d40cfd10 2023-07-06T21:40:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289203647 0 0) 0x7f43d40cf330 con 0x7f43d40b81c0 2023-07-06T21:40:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43d40d1830 con 0x7f43d40b81c0 2023-07-06T21:40:30.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43dab9c700 1 -- 172.21.15.84:0/2969912969 >> v1:172.21.15.107:6789/0 conn(0x7f43d40b81c0 legacy=0x7f43d40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43dab9c700 1 -- 172.21.15.84:0/2969912969 >> v1:172.21.15.84:6789/0 conn(0x7f43d40cfd10 legacy=0x7f43d40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43dab9c700 1 -- 172.21.15.84:0/2969912969 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d40d1550 con 0x7f43d40b1a80 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.771+0000 7f43dab9c700 1 --2- 172.21.15.84:0/2969912969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40b1a80 0x7f43d40d2a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009dc0 tx=0x7f43cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e0301dad6d35365 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc00e070 con 0x7f43d40b1a80 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc00bb20 con 0x7f43d40b1a80 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43d9399700 1 -- 172.21.15.84:0/2969912969 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc012430 con 0x7f43d40b1a80 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 -- 172.21.15.84:0/2969912969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40b1a80 msgr2=0x7f43d40d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/2969912969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40b1a80 0x7f43d40d2a90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009dc0 tx=0x7f43cc00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 -- 172.21.15.84:0/2969912969 shutdown_connections 2023-07-06T21:40:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/2969912969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40b1a80 0x7f43d40d2a90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 -- 172.21.15.84:0/2969912969 >> 172.21.15.84:0/2969912969 conn(0x7f43d406b2b0 msgr2=0x7f43d406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 -- 172.21.15.84:0/2969912969 shutdown_connections 2023-07-06T21:40:30.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.772+0000 7f43dc5ff700 1 -- 172.21.15.84:0/2969912969 wait complete. 2023-07-06T21:40:30.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.774+0000 7f43dc5ff700 1 Processor -- start 2023-07-06T21:40:30.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.774+0000 7f43dc5ff700 1 -- start start 2023-07-06T21:40:30.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dc5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 0x7f43d40cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dc5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43d40b81c0 0x7f43d41f1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dc5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40cfd10 0x7f43d40b7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dc5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40cc9d0 con 0x7f43d40b1a80 2023-07-06T21:40:30.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dc5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40c9cc0 con 0x7f43d40b81c0 2023-07-06T21:40:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dc5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40c9fa0 con 0x7f43d40cfd10 2023-07-06T21:40:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43da39b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 0x7f43d40cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43da39b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 0x7f43d40cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46098/0 (socket says 172.21.15.84:46098) 2023-07-06T21:40:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43da39b700 1 -- 172.21.15.84:0/1739529278 learned_addr learned my addr 172.21.15.84:0/1739529278 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43dab9c700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40cfd10 0x7f43d40b7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.775+0000 7f43d9b9a700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43d40b81c0 0x7f43d41f1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43da39b700 1 -- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40cfd10 msgr2=0x7f43d40b7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43da39b700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40cfd10 0x7f43d40b7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43da39b700 1 -- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43d40b81c0 msgr2=0x7f43d41f1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43da39b700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43d40b81c0 0x7f43d41f1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43da39b700 1 -- 172.21.15.84:0/1739529278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc009a70 con 0x7f43d40b1a80 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43dab9c700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40cfd10 0x7f43d40b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43da39b700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 0x7f43d40cad60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c520 tx=0x7f43d000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:30.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00052d0 con 0x7f43d40b1a80 2023-07-06T21:40:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d0005430 con 0x7f43d40b1a80 2023-07-06T21:40:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d00038c0 con 0x7f43d40b1a80 2023-07-06T21:40:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d40ca280 con 0x7f43d40b1a80 2023-07-06T21:40:30.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.776+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d40ca840 con 0x7f43d40b1a80 2023-07-06T21:40:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.777+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d404e1d0 con 0x7f43d40b1a80 2023-07-06T21:40:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.778+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f43d0003a20 con 0x7f43d40b1a80 2023-07-06T21:40:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.779+0000 7f43caffd700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43b806ace0 0x7f43b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:30.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.779+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f43d0088aa0 con 0x7f43d40b1a80 2023-07-06T21:40:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.780+0000 7f43d9b9a700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43b806ace0 0x7f43b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:30.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.780+0000 7f43d9b9a700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43b806ace0 0x7f43b806d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f43d41e8d40 tx=0x7f43c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.784+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f43d0053b10 con 0x7f43d40b1a80 2023-07-06T21:40:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.980+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43d40b4f30 con 0x7f43d40b1a80 2023-07-06T21:40:30.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.981+0000 7f43caffd700 1 -- 172.21.15.84:0/1739529278 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f43d0017020 con 0x7f43d40b1a80 2023-07-06T21:40:30.985 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:30.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43b806ace0 msgr2=0x7f43b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43b806ace0 0x7f43b806d190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f43d41e8d40 tx=0x7f43c4009040 comp rx=0 tx=0).stop 2023-07-06T21:40:30.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 msgr2=0x7f43d40cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:30.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 0x7f43d40cad60 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c520 tx=0x7f43d000c8e0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 shutdown_connections 2023-07-06T21:40:30.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f43b806ace0 0x7f43b806d190 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43d40cfd10 0x7f43d40b7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f43d40b1a80 0x7f43d40cad60 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 --2- 172.21.15.84:0/1739529278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43d40b81c0 0x7f43d41f1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:30.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 >> 172.21.15.84:0/1739529278 conn(0x7f43d406b2b0 msgr2=0x7f43d40c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:30.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 shutdown_connections 2023-07-06T21:40:30.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:30.983+0000 7f43dc5ff700 1 -- 172.21.15.84:0/1739529278 wait complete. 2023-07-06T21:40:31.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:31.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.204+0000 7fe9dbfff700 1 Processor -- start 2023-07-06T21:40:31.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.204+0000 7fe9dbfff700 1 -- start start 2023-07-06T21:40:31.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40cfd10 0x7fe9d40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9dbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe9d40b1a80 0x7fe9d40d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9dbfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d40d1830 con 0x7fe9d40b81c0 2023-07-06T21:40:31.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9dbfff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d40cf330 con 0x7fe9d40b1a80 2023-07-06T21:40:31.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9dbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40d1270 con 0x7fe9d40cfd10 2023-07-06T21:40:31.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9d9d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe9d40b81c0 0x7fe9d40cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46218/0 (socket says 172.21.15.84:46218) 2023-07-06T21:40:31.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.205+0000 7fe9d9d9b700 1 -- 172.21.15.84:0/782541883 learned_addr learned my addr 172.21.15.84:0/782541883 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:31.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9da59c700 1 --2- 172.21.15.84:0/782541883 >> v2:172.21.15.84:3300/0 conn(0x7fe9d40b1a80 0x7fe9d40d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9d959a700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40cfd10 0x7fe9d40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9d8d99700 1 -- 172.21.15.84:0/782541883 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218631772 0 0) 0x7fe9d40d1830 con 0x7fe9d40b81c0 2023-07-06T21:40:31.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9d8d99700 1 -- 172.21.15.84:0/782541883 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c00035b0 con 0x7fe9d40b81c0 2023-07-06T21:40:31.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9da59c700 1 -- 172.21.15.84:0/782541883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40cfd10 msgr2=0x7fe9d40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:31.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9da59c700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40cfd10 0x7fe9d40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9da59c700 1 -- 172.21.15.84:0/782541883 >> v1:172.21.15.84:6789/0 conn(0x7fe9d40b81c0 legacy=0x7fe9d40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9da59c700 1 -- 172.21.15.84:0/782541883 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40d1550 con 0x7fe9d40b1a80 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9d959a700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40cfd10 0x7fe9d40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9da59c700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40d28c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009dc0 tx=0x7fe9cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb29656e291c121 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.206+0000 7fe9d8d99700 1 -- 172.21.15.84:0/782541883 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00e070 con 0x7fe9d40b1a80 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9d8d99700 1 -- 172.21.15.84:0/782541883 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9cc00bae0 con 0x7fe9d40b1a80 2023-07-06T21:40:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9d8d99700 1 -- 172.21.15.84:0/782541883 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc00be90 con 0x7fe9d40b1a80 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 -- 172.21.15.84:0/782541883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 msgr2=0x7fe9d40d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40d28c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009dc0 tx=0x7fe9cc00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 -- 172.21.15.84:0/782541883 shutdown_connections 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40d28c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/782541883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40cfd10 0x7fe9d40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 -- 172.21.15.84:0/782541883 >> 172.21.15.84:0/782541883 conn(0x7fe9d406b2b0 msgr2=0x7fe9d406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:31.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 -- 172.21.15.84:0/782541883 shutdown_connections 2023-07-06T21:40:31.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.207+0000 7fe9dbfff700 1 -- 172.21.15.84:0/782541883 wait complete. 2023-07-06T21:40:31.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.209+0000 7fe9dbfff700 1 Processor -- start 2023-07-06T21:40:31.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.209+0000 7fe9dbfff700 1 -- start start 2023-07-06T21:40:31.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40b81c0 0x7fe9d41f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9dbfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d40cfd10 0x7fe9d40ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9dbfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40ca030 con 0x7fe9d40b1a80 2023-07-06T21:40:31.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9dbfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40b3100 con 0x7fe9d40cfd10 2023-07-06T21:40:31.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9dbfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d40b33e0 con 0x7fe9d40b81c0 2023-07-06T21:40:31.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9d9d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9d9d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46108/0 (socket says 172.21.15.84:46108) 2023-07-06T21:40:31.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.210+0000 7fe9d9d9b700 1 -- 172.21.15.84:0/2365863550 learned_addr learned my addr 172.21.15.84:0/2365863550 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:31.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9d9d9b700 1 -- 172.21.15.84:0/2365863550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40b81c0 msgr2=0x7fe9d41f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9d9d9b700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40b81c0 0x7fe9d41f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9d9d9b700 1 -- 172.21.15.84:0/2365863550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d40cfd10 msgr2=0x7fe9d40ca650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9d9d9b700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d40cfd10 0x7fe9d40ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9d9d9b700 1 -- 172.21.15.84:0/2365863550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc009a70 con 0x7fe9d40b1a80 2023-07-06T21:40:31.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9d9d9b700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40b3be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000c700 tx=0x7fe9d000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:31.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0005d70 con 0x7fe9d40b1a80 2023-07-06T21:40:31.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d00070d0 con 0x7fe9d40b1a80 2023-07-06T21:40:31.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d40b36c0 con 0x7fe9d40b1a80 2023-07-06T21:40:31.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0007440 con 0x7fe9d40b1a80 2023-07-06T21:40:31.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.211+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d40c9cc0 con 0x7fe9d40b1a80 2023-07-06T21:40:31.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.212+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d40b3900 con 0x7fe9d40b1a80 2023-07-06T21:40:31.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.213+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe9d00075a0 con 0x7fe9d40b1a80 2023-07-06T21:40:31.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.214+0000 7fe9ca7fc700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe9b806ace0 0x7fe9b806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.214+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fe9d00889f0 con 0x7fe9d40b1a80 2023-07-06T21:40:31.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.214+0000 7fe9d959a700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe9b806ace0 0x7fe9b806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.215+0000 7fe9d959a700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe9b806ace0 0x7fe9b806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006f10 tx=0x7fe9c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:31.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.218+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe9d40b3900 con 0x7fe9d40b1a80 2023-07-06T21:40:31.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.415+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9d40b3900 con 0x7fe9d40b1a80 2023-07-06T21:40:31.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.416+0000 7fe9ca7fc700 1 -- 172.21.15.84:0/2365863550 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9d40b3900 con 0x7fe9d40b1a80 2023-07-06T21:40:31.421 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:31.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe9b806ace0 msgr2=0x7fe9b806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe9b806ace0 0x7fe9b806d190 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006f10 tx=0x7fe9c4008040 comp rx=0 tx=0).stop 2023-07-06T21:40:31.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 msgr2=0x7fe9d40b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40b3be0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000c700 tx=0x7fe9d000cac0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 shutdown_connections 2023-07-06T21:40:31.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe9b806ace0 0x7fe9b806d190 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d40cfd10 0x7fe9d40ca650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9d40b1a80 0x7fe9d40b3be0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 --2- 172.21.15.84:0/2365863550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d40b81c0 0x7fe9d41f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 >> 172.21.15.84:0/2365863550 conn(0x7fe9d406b2b0 msgr2=0x7fe9d40b6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:31.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 shutdown_connections 2023-07-06T21:40:31.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.418+0000 7fe9dbfff700 1 -- 172.21.15.84:0/2365863550 wait complete. 2023-07-06T21:40:31.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:31.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.632+0000 7fb5695ff700 1 Processor -- start 2023-07-06T21:40:31.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.632+0000 7fb5695ff700 1 -- start start 2023-07-06T21:40:31.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5695ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb564069e90 0x7fb56406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5695ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 0x7fb56406a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5695ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5640c94b0 con 0x7fb5640cf280 2023-07-06T21:40:31.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5695ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56406d350 con 0x7fb564069e90 2023-07-06T21:40:31.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5695ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56406a7a0 con 0x7fb56406dde0 2023-07-06T21:40:31.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5637fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb5640cf280 0x7fb5640b1650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46230/0 (socket says 172.21.15.84:46230) 2023-07-06T21:40:31.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5637fe700 1 -- 172.21.15.84:0/2865548425 learned_addr learned my addr 172.21.15.84:0/2865548425 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:31.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb562ffd700 1 --2- 172.21.15.84:0/2865548425 >> v2:172.21.15.107:3300/0 conn(0x7fb564069e90 0x7fb56406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb55bfff700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 0x7fb56406a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832552460 0 0) 0x7fb5640c94b0 con 0x7fb5640cf280 2023-07-06T21:40:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.634+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5500035b0 con 0x7fb5640cf280 2023-07-06T21:40:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471873068 0 0) 0x7fb5500035b0 con 0x7fb5640cf280 2023-07-06T21:40:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 msgr2=0x7fb56406a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:31.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 0x7fb56406a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 >> v2:172.21.15.107:3300/0 conn(0x7fb564069e90 msgr2=0x7fb56406ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 --2- 172.21.15.84:0/2865548425 >> v2:172.21.15.107:3300/0 conn(0x7fb564069e90 0x7fb56406ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640ca9b0 con 0x7fb5640cf280 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fb55c0019f0 con 0x7fb5640cf280 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 >> v1:172.21.15.84:6789/0 conn(0x7fb5640cf280 legacy=0x7fb5640b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 0x7fb550003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb550003f80 0x7fb550004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550004d90 0x7fb550005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56406d350 con 0x7fb550003f80 2023-07-06T21:40:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56406a7a0 con 0x7fb550004d90 2023-07-06T21:40:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55000ba40 con 0x7fb56406dde0 2023-07-06T21:40:31.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb55bfff700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb550003f80 0x7fb550004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb5637fe700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550004d90 0x7fb550005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.635+0000 7fb562ffd700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 0x7fb550003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb55bfff700 1 -- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 msgr2=0x7fb550003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:31.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb55bfff700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb56406dde0 0x7fb550003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb55bfff700 1 -- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550004d90 msgr2=0x7fb550005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:31.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb55bfff700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550004d90 0x7fb550005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb55bfff700 1 -- 172.21.15.84:0/2865548425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5500035b0 con 0x7fb550003f80 2023-07-06T21:40:31.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb55bfff700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb550003f80 0x7fb550004850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009b60 tx=0x7fb550005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ecab519417a2e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:31.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00e660 con 0x7fb550003f80 2023-07-06T21:40:31.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00e7c0 con 0x7fb550003f80 2023-07-06T21:40:31.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb5627fc700 1 -- 172.21.15.84:0/2865548425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00eb30 con 0x7fb550003f80 2023-07-06T21:40:31.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb5695ff700 1 -- 172.21.15.84:0/2865548425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb550003f80 msgr2=0x7fb550004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb5695ff700 1 --2- 172.21.15.84:0/2865548425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb550003f80 0x7fb550004850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009b60 tx=0x7fb550005880 comp rx=0 tx=0).stop 2023-07-06T21:40:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.636+0000 7fb5637fe700 1 -- 172.21.15.84:0/2865548425 reap_dead start 2023-07-06T21:40:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.637+0000 7fb5695ff700 1 -- 172.21.15.84:0/2865548425 shutdown_connections 2023-07-06T21:40:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.637+0000 7fb5695ff700 1 -- 172.21.15.84:0/2865548425 >> 172.21.15.84:0/2865548425 conn(0x7fb5640cbf10 msgr2=0x7fb5640ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.637+0000 7fb5695ff700 1 -- 172.21.15.84:0/2865548425 shutdown_connections 2023-07-06T21:40:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.637+0000 7fb5695ff700 1 -- 172.21.15.84:0/2865548425 wait complete. 2023-07-06T21:40:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.638+0000 7fb5695ff700 1 Processor -- start 2023-07-06T21:40:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.639+0000 7fb5695ff700 1 -- start start 2023-07-06T21:40:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5695ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550003f80 0x7fb5641518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5695ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 0x7fb5640cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5695ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb564151fe0 0x7fb5640ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5695ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb564151de0 con 0x7fb5640cc6e0 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5695ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640ccc50 con 0x7fb550003f80 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5695ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5640ccf30 con 0x7fb564151fe0 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 0x7fb5640cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 0x7fb5640cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46126/0 (socket says 172.21.15.84:46126) 2023-07-06T21:40:31.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 -- 172.21.15.84:0/915653448 learned_addr learned my addr 172.21.15.84:0/915653448 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb562ffd700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550003f80 0x7fb5641518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5637fe700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb564151fe0 0x7fb5640ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 -- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb564151fe0 msgr2=0x7fb5640ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb564151fe0 0x7fb5640ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 -- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550003f80 msgr2=0x7fb5641518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550003f80 0x7fb5641518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 -- 172.21.15.84:0/915653448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5640cd210 con 0x7fb5640cc6e0 2023-07-06T21:40:31.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb5637fe700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb564151fe0 0x7fb5640ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.640+0000 7fb55bfff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 0x7fb5640cdcf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb54c000c00 tx=0x7fb54c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.641+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00e660 con 0x7fb5640cc6e0 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.641+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c003ad0 con 0x7fb5640cc6e0 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.641+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb564151de0 con 0x7fb5640cc6e0 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.641+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c003e80 con 0x7fb5640cc6e0 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.641+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5640cd7d0 con 0x7fb5640cc6e0 2023-07-06T21:40:31.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.642+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5640cd990 con 0x7fb5640cc6e0 2023-07-06T21:40:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.643+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb54c013470 con 0x7fb5640cc6e0 2023-07-06T21:40:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.644+0000 7fb55b7fe700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb54406ace0 0x7fb54406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.644+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb54c08ab80 con 0x7fb5640cc6e0 2023-07-06T21:40:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.644+0000 7fb562ffd700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb54406ace0 0x7fb54406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:31.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.644+0000 7fb562ffd700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb54406ace0 0x7fb54406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5640abfb0 tx=0x7fb554008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:31.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.648+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb54c055da0 con 0x7fb5640cc6e0 2023-07-06T21:40:31.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.843+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb5640cdb40 con 0x7fb5640cc6e0 2023-07-06T21:40:31.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.844+0000 7fb55b7fe700 1 -- 172.21.15.84:0/915653448 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb54c0593c0 con 0x7fb5640cc6e0 2023-07-06T21:40:31.847 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:31.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb54406ace0 msgr2=0x7fb54406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb54406ace0 0x7fb54406d190 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb5640abfb0 tx=0x7fb554008040 comp rx=0 tx=0).stop 2023-07-06T21:40:31.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 msgr2=0x7fb5640cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 0x7fb5640cdcf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb54c000c00 tx=0x7fb54c0047c0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 shutdown_connections 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb54406ace0 0x7fb54406d190 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb564151fe0 0x7fb5640ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb550003f80 0x7fb5641518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 --2- 172.21.15.84:0/915653448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5640cc6e0 0x7fb5640cdcf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:31.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 >> 172.21.15.84:0/915653448 conn(0x7fb5640cbf10 msgr2=0x7fb564159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:31.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 shutdown_connections 2023-07-06T21:40:31.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:31.846+0000 7fb5695ff700 1 -- 172.21.15.84:0/915653448 wait complete. 2023-07-06T21:40:31.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.061+0000 7fb8be9ff700 1 Processor -- start 2023-07-06T21:40:32.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.061+0000 7fb8be9ff700 1 -- start start 2023-07-06T21:40:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8be9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8be9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80cd910 con 0x7fb8b80b83c0 2023-07-06T21:40:32.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8be9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80d0a80 con 0x7fb8b80d3710 2023-07-06T21:40:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8be9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80b8790 con 0x7fb8b80cfc60 2023-07-06T21:40:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8b7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb8b80b83c0 0x7fb8b80cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46240/0 (socket says 172.21.15.84:46240) 2023-07-06T21:40:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8b7fff700 1 -- 172.21.15.84:0/3407464653 learned_addr learned my addr 172.21.15.84:0/3407464653 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:32.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.062+0000 7fb8bcf9c700 1 --2- 172.21.15.84:0/3407464653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 333360589 0 0) 0x7fb8a8001110 con 0x7fb8b80b83c0 2023-07-06T21:40:32.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a00035b0 con 0x7fb8b80b83c0 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936738937 0 0) 0x7fb8b80d0a80 con 0x7fb8b80d3710 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a8001110 con 0x7fb8b80d3710 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/3407464653 >> v1:172.21.15.107:6789/0 conn(0x7fb8b80d3710 legacy=0x7fb8b80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/3407464653 >> v1:172.21.15.84:6789/0 conn(0x7fb8b80b83c0 legacy=0x7fb8b80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/3407464653 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b80b4e70 con 0x7fb8b80cfc60 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8bcf9c700 1 --2- 172.21.15.84:0/3407464653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8167010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009dc0 tx=0x7fb8b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8650d7353180a9a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b000e070 con 0x7fb8b80cfc60 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b000bb20 con 0x7fb8b80cfc60 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.063+0000 7fb8b6ffd700 1 -- 172.21.15.84:0/3407464653 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0012430 con 0x7fb8b80cfc60 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 -- 172.21.15.84:0/3407464653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 msgr2=0x7fb8b8167010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/3407464653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8167010 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb8b0009dc0 tx=0x7fb8b000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 -- 172.21.15.84:0/3407464653 shutdown_connections 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/3407464653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8167010 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 -- 172.21.15.84:0/3407464653 >> 172.21.15.84:0/3407464653 conn(0x7fb8b806b350 msgr2=0x7fb8b806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:32.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 -- 172.21.15.84:0/3407464653 shutdown_connections 2023-07-06T21:40:32.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.064+0000 7fb8be9ff700 1 -- 172.21.15.84:0/3407464653 wait complete. 2023-07-06T21:40:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.066+0000 7fb8be9ff700 1 Processor -- start 2023-07-06T21:40:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.066+0000 7fb8be9ff700 1 -- start start 2023-07-06T21:40:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8be9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b80b83c0 0x7fb8b81544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8be9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8be9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 0x7fb8b81f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8be9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b815e300 con 0x7fb8b80d3710 2023-07-06T21:40:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8be9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b815c930 con 0x7fb8b80b83c0 2023-07-06T21:40:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8be9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b815e550 con 0x7fb8b80cfc60 2023-07-06T21:40:32.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8bcf9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 0x7fb8b81f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8bcf9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 0x7fb8b81f2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46134/0 (socket says 172.21.15.84:46134) 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/1999282004 learned_addr learned my addr 172.21.15.84:0/1999282004 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8b7fff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b80b83c0 0x7fb8b81544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.067+0000 7fb8b77fe700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 msgr2=0x7fb8b8149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8bcf9c700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b80b83c0 msgr2=0x7fb8b81544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8bcf9c700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b80b83c0 0x7fb8b81544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8bcf9c700 1 -- 172.21.15.84:0/1999282004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b0009a70 con 0x7fb8b80d3710 2023-07-06T21:40:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8b7fff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b80b83c0 0x7fb8b81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8b77fe700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:32.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.068+0000 7fb8bcf9c700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 0x7fb8b81f2170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000b370 tx=0x7fb8b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.069+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b001b810 con 0x7fb8b80d3710 2023-07-06T21:40:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.069+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b0020070 con 0x7fb8b80d3710 2023-07-06T21:40:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.069+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b001bbf0 con 0x7fb8b80d3710 2023-07-06T21:40:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.069+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b815e830 con 0x7fb8b80d3710 2023-07-06T21:40:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.070+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b815edf0 con 0x7fb8b80d3710 2023-07-06T21:40:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.071+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b815caa0 con 0x7fb8b80d3710 2023-07-06T21:40:32.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.072+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb8b001bd50 con 0x7fb8b80d3710 2023-07-06T21:40:32.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.072+0000 7fb8b4ff9700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb8a406ace0 0x7fb8a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.072+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb8b0022070 con 0x7fb8b80d3710 2023-07-06T21:40:32.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.073+0000 7fb8b7fff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb8a406ace0 0x7fb8a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.073+0000 7fb8b7fff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb8a406ace0 0x7fb8a406d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8000f60 tx=0x7fb8a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.077+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb8b00568e0 con 0x7fb8b80d3710 2023-07-06T21:40:32.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.292+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb8b815b470 con 0x7fb8b80d3710 2023-07-06T21:40:32.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.292+0000 7fb8b4ff9700 1 -- 172.21.15.84:0/1999282004 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb8b0059f00 con 0x7fb8b80d3710 2023-07-06T21:40:32.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb8a406ace0 msgr2=0x7fb8a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb8a406ace0 0x7fb8a406d190 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8000f60 tx=0x7fb8a8009040 comp rx=0 tx=0).stop 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 msgr2=0x7fb8b81f2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 0x7fb8b81f2170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb8b000b370 tx=0x7fb8b0004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 shutdown_connections 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb8a406ace0 0x7fb8a406d190 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb8b80d3710 0x7fb8b81f2170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8b80b83c0 0x7fb8b81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 --2- 172.21.15.84:0/1999282004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8b80cfc60 0x7fb8b8149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.294+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 >> 172.21.15.84:0/1999282004 conn(0x7fb8b806b350 msgr2=0x7fb8b80b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:32.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.295+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 shutdown_connections 2023-07-06T21:40:32.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.295+0000 7fb8be9ff700 1 -- 172.21.15.84:0/1999282004 wait complete. 2023-07-06T21:40:32.303 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:32.316 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:40:32.316 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:40:32.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:32.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.555+0000 7f2b973ff700 1 Processor -- start 2023-07-06T21:40:32.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.555+0000 7f2b973ff700 1 -- start start 2023-07-06T21:40:32.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b973ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b900d3710 0x7f2b900cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b973ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b900cd910 con 0x7f2b900cfa90 2023-07-06T21:40:32.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b973ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b900d0a80 con 0x7f2b900b83c0 2023-07-06T21:40:32.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b973ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900cff10 con 0x7f2b900d3710 2023-07-06T21:40:32.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b9599c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2b900cfa90 0x7f2b900d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46250/0 (socket says 172.21.15.84:46250) 2023-07-06T21:40:32.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b9499a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2b900d3710 0x7f2b900cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.556+0000 7f2b9599c700 1 -- 172.21.15.84:0/1974530723 learned_addr learned my addr 172.21.15.84:0/1974530723 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:32.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b87fff700 1 -- 172.21.15.84:0/1974530723 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420785753 0 0) 0x7f2b88001490 con 0x7f2b900cfa90 2023-07-06T21:40:32.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b9499a700 1 -- 172.21.15.84:0/1974530723 >> v1:172.21.15.107:6789/0 conn(0x7f2b900b83c0 legacy=0x7f2b900cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:32.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b9499a700 1 -- 172.21.15.84:0/1974530723 >> v1:172.21.15.84:6789/0 conn(0x7f2b900cfa90 legacy=0x7f2b900d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b9499a700 1 -- 172.21.15.84:0/1974530723 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900b8790 con 0x7f2b900d3710 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b9499a700 1 --2- 172.21.15.84:0/1974530723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900d3710 0x7f2b900cf550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009840 tx=0x7f2b80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a547ab5375f29b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b87fff700 1 -- 172.21.15.84:0/1974530723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80011830 con 0x7f2b900d3710 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b87fff700 1 -- 172.21.15.84:0/1974530723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b80011990 con 0x7f2b900d3710 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.557+0000 7f2b87fff700 1 -- 172.21.15.84:0/1974530723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b80011d20 con 0x7f2b900d3710 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 -- 172.21.15.84:0/1974530723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900d3710 msgr2=0x7f2b900cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1974530723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900d3710 0x7f2b900cf550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009840 tx=0x7f2b80009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:32.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 -- 172.21.15.84:0/1974530723 shutdown_connections 2023-07-06T21:40:32.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1974530723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900d3710 0x7f2b900cf550 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 -- 172.21.15.84:0/1974530723 >> 172.21.15.84:0/1974530723 conn(0x7f2b9006b350 msgr2=0x7f2b9006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:32.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 -- 172.21.15.84:0/1974530723 shutdown_connections 2023-07-06T21:40:32.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.558+0000 7f2b973ff700 1 -- 172.21.15.84:0/1974530723 wait complete. 2023-07-06T21:40:32.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.560+0000 7f2b973ff700 1 Processor -- start 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.560+0000 7f2b973ff700 1 -- start start 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b973ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b900b83c0 0x7f2b9015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b973ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 0x7f2b900b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b973ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b900d3710 0x7f2b900b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b973ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9015cb40 con 0x7f2b900cfa90 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b973ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9015d9a0 con 0x7f2b900b83c0 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b973ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9015cd90 con 0x7f2b900d3710 2023-07-06T21:40:32.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 0x7f2b900b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 0x7f2b900b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46152/0 (socket says 172.21.15.84:46152) 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 -- 172.21.15.84:0/1722984269 learned_addr learned my addr 172.21.15.84:0/1722984269 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 -- 172.21.15.84:0/1722984269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b900d3710 msgr2=0x7f2b900b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b900d3710 0x7f2b900b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 -- 172.21.15.84:0/1722984269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b900b83c0 msgr2=0x7f2b9015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b900b83c0 0x7f2b9015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.561+0000 7f2b9499a700 1 -- 172.21.15.84:0/1722984269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800094a0 con 0x7f2b900cfa90 2023-07-06T21:40:32.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.562+0000 7f2b9499a700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 0x7f2b900b3180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2b80014040 tx=0x7f2b80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.562+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8001a900 con 0x7f2b900cfa90 2023-07-06T21:40:32.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.562+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8001aa60 con 0x7f2b900cfa90 2023-07-06T21:40:32.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.562+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8001add0 con 0x7f2b900cfa90 2023-07-06T21:40:32.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.562+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9015d070 con 0x7f2b900cfa90 2023-07-06T21:40:32.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.562+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9015d630 con 0x7f2b900cfa90 2023-07-06T21:40:32.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.563+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9015d7f0 con 0x7f2b900cfa90 2023-07-06T21:40:32.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.564+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2b80022070 con 0x7f2b900cfa90 2023-07-06T21:40:32.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.565+0000 7f2b85ffb700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b7406ace0 0x7f2b7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.565+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2b8008a360 con 0x7f2b900cfa90 2023-07-06T21:40:32.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.565+0000 7f2b9519b700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b7406ace0 0x7f2b7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.566+0000 7f2b9519b700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b7406ace0 0x7f2b7406d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000f60 tx=0x7f2b8c007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.569+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2b80055510 con 0x7f2b900cfa90 2023-07-06T21:40:32.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.765+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b900ce760 con 0x7f2b900cfa90 2023-07-06T21:40:32.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.766+0000 7f2b85ffb700 1 -- 172.21.15.84:0/1722984269 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f2b80058b30 con 0x7f2b900cfa90 2023-07-06T21:40:32.770 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:32.770 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:32.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b7406ace0 msgr2=0x7f2b7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b7406ace0 0x7f2b7406d190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c000f60 tx=0x7f2b8c007a40 comp rx=0 tx=0).stop 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 msgr2=0x7f2b900b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 0x7f2b900b3180 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2b80014040 tx=0x7f2b80004750 comp rx=0 tx=0).stop 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 shutdown_connections 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b7406ace0 0x7f2b7406d190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b900d3710 0x7f2b900b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b900b83c0 0x7f2b9015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 --2- 172.21.15.84:0/1722984269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2b900cfa90 0x7f2b900b3180 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 >> 172.21.15.84:0/1722984269 conn(0x7f2b9006b350 msgr2=0x7f2b900b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:32.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 shutdown_connections 2023-07-06T21:40:32.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.769+0000 7f2b973ff700 1 -- 172.21.15.84:0/1722984269 wait complete. 2023-07-06T21:40:32.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:32.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:32.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.985+0000 7f411b59d700 1 Processor -- start 2023-07-06T21:40:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.985+0000 7f411b59d700 1 -- start start 2023-07-06T21:40:32.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411b59d700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41140b2e40 0x7f41140cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411b59d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41140d0c90 0x7f41140b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411b59d700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41140b3e50 con 0x7f41140cfb70 2023-07-06T21:40:32.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411b59d700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411415ca50 con 0x7f41140d0c90 2023-07-06T21:40:32.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411b59d700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41140b3b70 con 0x7f41140b2e40 2023-07-06T21:40:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411ad9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f41140d0c90 0x7f41140b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f4119d9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f41140cfb70 0x7f41140d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46454/0 (socket says 172.21.15.84:46454) 2023-07-06T21:40:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f4119d9a700 1 -- 172.21.15.84:0/4256271077 learned_addr learned my addr 172.21.15.84:0/4256271077 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411a59b700 1 --2- 172.21.15.84:0/4256271077 >> v2:172.21.15.107:3300/0 conn(0x7f41140b2e40 0x7f41140cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f4119599700 1 -- 172.21.15.84:0/4256271077 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466736069 0 0) 0x7f41140b3e50 con 0x7f41140cfb70 2023-07-06T21:40:32.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f4119599700 1 -- 172.21.15.84:0/4256271077 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41000035b0 con 0x7f41140cfb70 2023-07-06T21:40:32.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411ad9c700 1 -- 172.21.15.84:0/4256271077 >> v2:172.21.15.107:3300/0 conn(0x7f41140b2e40 msgr2=0x7f41140cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:32.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411ad9c700 1 --2- 172.21.15.84:0/4256271077 >> v2:172.21.15.107:3300/0 conn(0x7f41140b2e40 0x7f41140cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411ad9c700 1 -- 172.21.15.84:0/4256271077 >> v1:172.21.15.84:6789/0 conn(0x7f41140cfb70 legacy=0x7f41140d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.987+0000 7f411ad9c700 1 -- 172.21.15.84:0/4256271077 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140ceb50 con 0x7f41140d0c90 2023-07-06T21:40:32.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.988+0000 7f411ad9c700 1 --2- 172.21.15.84:0/4256271077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140d0c90 0x7f41140b5800 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4114066620 tx=0x7f410c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc70939aebba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:32.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.988+0000 7f4119599700 1 -- 172.21.15.84:0/4256271077 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c00e070 con 0x7f41140d0c90 2023-07-06T21:40:32.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.988+0000 7f4119599700 1 -- 172.21.15.84:0/4256271077 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410c00bb20 con 0x7f41140d0c90 2023-07-06T21:40:32.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.988+0000 7f4119599700 1 -- 172.21.15.84:0/4256271077 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410c00be90 con 0x7f41140d0c90 2023-07-06T21:40:32.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.988+0000 7f411b59d700 1 -- 172.21.15.84:0/4256271077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140d0c90 msgr2=0x7f41140b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:32.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.988+0000 7f411b59d700 1 --2- 172.21.15.84:0/4256271077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140d0c90 0x7f41140b5800 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4114066620 tx=0x7f410c00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:32.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.989+0000 7f411b59d700 1 -- 172.21.15.84:0/4256271077 shutdown_connections 2023-07-06T21:40:32.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.989+0000 7f411b59d700 1 --2- 172.21.15.84:0/4256271077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140d0c90 0x7f41140b5800 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.989+0000 7f411b59d700 1 --2- 172.21.15.84:0/4256271077 >> v2:172.21.15.107:3300/0 conn(0x7f41140b2e40 0x7f41140cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:32.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.989+0000 7f411b59d700 1 -- 172.21.15.84:0/4256271077 >> 172.21.15.84:0/4256271077 conn(0x7f411409bcf0 msgr2=0x7f41140d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:32.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.989+0000 7f411b59d700 1 -- 172.21.15.84:0/4256271077 shutdown_connections 2023-07-06T21:40:32.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.989+0000 7f411b59d700 1 -- 172.21.15.84:0/4256271077 wait complete. 2023-07-06T21:40:33.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.990+0000 7f411b59d700 1 Processor -- start 2023-07-06T21:40:33.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 -- start start 2023-07-06T21:40:33.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41140b2e40 0x7f41141e3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 0x7f411414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41140d0c90 0x7f41140c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411414b940 con 0x7f41140cfb70 2023-07-06T21:40:33.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41141e40f0 con 0x7f41140d0c90 2023-07-06T21:40:33.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.991+0000 7f411b59d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411414e820 con 0x7f41140b2e40 2023-07-06T21:40:33.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 0x7f411414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 0x7f411414dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52242/0 (socket says 172.21.15.84:52242) 2023-07-06T21:40:33.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 -- 172.21.15.84:0/1095578667 learned_addr learned my addr 172.21.15.84:0/1095578667 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:33.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f411ad9c700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41140d0c90 0x7f41140c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f411a59b700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41140b2e40 0x7f41141e3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 -- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41140b2e40 msgr2=0x7f41141e3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41140b2e40 0x7f41141e3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 -- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41140d0c90 msgr2=0x7f41140c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41140d0c90 0x7f41140c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 -- 172.21.15.84:0/1095578667 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f410c009a70 con 0x7f41140cfb70 2023-07-06T21:40:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f411ad9c700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41140d0c90 0x7f41140c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f4119d9a700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 0x7f411414dc20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f410400c450 tx=0x7f410400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.992+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104005d70 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.993+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41040070d0 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.993+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41141e66e0 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.993+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104007440 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.993+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41141e6ca0 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.994+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41140d15f0 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.995+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41040075a0 con 0x7f41140cfb70 2023-07-06T21:40:33.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.996+0000 7f410affd700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40f806ace0 0x7f40f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.996+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4104088840 con 0x7f41140cfb70 2023-07-06T21:40:33.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.996+0000 7f411a59b700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40f806ace0 0x7f40f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:32.996+0000 7f411a59b700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40f806ace0 0x7f40f806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4110009ab0 tx=0x7f4110008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.000+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4104053950 con 0x7f41140cfb70 2023-07-06T21:40:33.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.198+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f411404e1d0 con 0x7f41140cfb70 2023-07-06T21:40:33.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.199+0000 7f410affd700 1 -- 172.21.15.84:0/1095578667 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4104056f70 con 0x7f41140cfb70 2023-07-06T21:40:33.202 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:33.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40f806ace0 msgr2=0x7f40f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40f806ace0 0x7f40f806d190 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4110009ab0 tx=0x7f4110008040 comp rx=0 tx=0).stop 2023-07-06T21:40:33.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 msgr2=0x7f411414dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 0x7f411414dc20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f410400c450 tx=0x7f410400c810 comp rx=0 tx=0).stop 2023-07-06T21:40:33.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 shutdown_connections 2023-07-06T21:40:33.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40f806ace0 0x7f40f806d190 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41140d0c90 0x7f41140c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41140b2e40 0x7f41141e3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 --2- 172.21.15.84:0/1095578667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41140cfb70 0x7f411414dc20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 >> 172.21.15.84:0/1095578667 conn(0x7f411409bcf0 msgr2=0x7f41140d0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:33.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 shutdown_connections 2023-07-06T21:40:33.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.201+0000 7f411b59d700 1 -- 172.21.15.84:0/1095578667 wait complete. 2023-07-06T21:40:33.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:33.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.414+0000 7f7661fff700 1 Processor -- start 2023-07-06T21:40:33.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.415+0000 7f7661fff700 1 -- start start 2023-07-06T21:40:33.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f7661fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f765c0cfd10 0x7f765c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f7661fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f765c0b1a80 0x7f765c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f7661fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c0d1830 con 0x7f765c0b81c0 2023-07-06T21:40:33.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f7661fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0cf330 con 0x7f765c0b1a80 2023-07-06T21:40:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f7661fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c0d1270 con 0x7f765c0cfd10 2023-07-06T21:40:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f765c0b1a80 0x7f765c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f765c0b1a80 0x7f765c0d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52246/0 (socket says 172.21.15.84:52246) 2023-07-06T21:40:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 -- 172.21.15.84:0/811148135 learned_addr learned my addr 172.21.15.84:0/811148135 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765affd700 1 --2- 172.21.15.84:0/811148135 >> v2:172.21.15.107:3300/0 conn(0x7f765c0cfd10 0x7f765c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 -- 172.21.15.84:0/811148135 >> v2:172.21.15.107:3300/0 conn(0x7f765c0cfd10 msgr2=0x7f765c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 --2- 172.21.15.84:0/811148135 >> v2:172.21.15.107:3300/0 conn(0x7f765c0cfd10 0x7f765c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 -- 172.21.15.84:0/811148135 >> v1:172.21.15.107:6789/0 conn(0x7f765c0b81c0 legacy=0x7f765c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.416+0000 7f765bfff700 1 -- 172.21.15.84:0/811148135 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0d1550 con 0x7f765c0b1a80 2023-07-06T21:40:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.417+0000 7f765bfff700 1 --2- 172.21.15.84:0/811148135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b1a80 0x7f765c0d28c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f765c0d0770 tx=0x7f765400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5660107ea036df43 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.417+0000 7f765a7fc700 1 -- 172.21.15.84:0/811148135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400e070 con 0x7f765c0b1a80 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.417+0000 7f765a7fc700 1 -- 172.21.15.84:0/811148135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765400b9c0 con 0x7f765c0b1a80 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.417+0000 7f765a7fc700 1 -- 172.21.15.84:0/811148135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400bd70 con 0x7f765c0b1a80 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 -- 172.21.15.84:0/811148135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b1a80 msgr2=0x7f765c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 --2- 172.21.15.84:0/811148135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b1a80 0x7f765c0d28c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f765c0d0770 tx=0x7f765400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 -- 172.21.15.84:0/811148135 shutdown_connections 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 --2- 172.21.15.84:0/811148135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b1a80 0x7f765c0d28c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 --2- 172.21.15.84:0/811148135 >> v2:172.21.15.107:3300/0 conn(0x7f765c0cfd10 0x7f765c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 -- 172.21.15.84:0/811148135 >> 172.21.15.84:0/811148135 conn(0x7f765c06b2b0 msgr2=0x7f765c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:33.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 -- 172.21.15.84:0/811148135 shutdown_connections 2023-07-06T21:40:33.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.418+0000 7f7661fff700 1 -- 172.21.15.84:0/811148135 wait complete. 2023-07-06T21:40:33.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.419+0000 7f7661fff700 1 Processor -- start 2023-07-06T21:40:33.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.420+0000 7f7661fff700 1 -- start start 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.420+0000 7f7661fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f765c0b1a80 0x7f765c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f7661fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 0x7f765c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f7661fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0cfd10 0x7f765c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f7661fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0b79f0 con 0x7f765c0b81c0 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f7661fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0caf10 con 0x7f765c0cfd10 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f7661fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c149510 con 0x7f765c0b1a80 2023-07-06T21:40:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 0x7f765c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 0x7f765c157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52262/0 (socket says 172.21.15.84:52262) 2023-07-06T21:40:33.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 -- 172.21.15.84:0/2970753002 learned_addr learned my addr 172.21.15.84:0/2970753002 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:33.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765bfff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0cfd10 0x7f765c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765b7fe700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f765c0b1a80 0x7f765c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 -- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f765c0b1a80 msgr2=0x7f765c148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f765c0b1a80 0x7f765c148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 -- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0cfd10 msgr2=0x7f765c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0cfd10 0x7f765c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 -- 172.21.15.84:0/2970753002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7654009b30 con 0x7f765c0b81c0 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.421+0000 7f765affd700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 0x7f765c157300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f765000d800 tx=0x7f765000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.422+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76500125f0 con 0x7f765c0b81c0 2023-07-06T21:40:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.422+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7650012750 con 0x7f765c0b81c0 2023-07-06T21:40:33.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.422+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7650012ae0 con 0x7f765c0b81c0 2023-07-06T21:40:33.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.422+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c0cb1b0 con 0x7f765c0b81c0 2023-07-06T21:40:33.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.422+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c0cb770 con 0x7f765c0b81c0 2023-07-06T21:40:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.423+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c0cb900 con 0x7f765c0b81c0 2023-07-06T21:40:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.424+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7650012c40 con 0x7f765c0b81c0 2023-07-06T21:40:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.425+0000 7f7643fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f764806ace0 0x7f764806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.425+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f76500898a0 con 0x7f765c0b81c0 2023-07-06T21:40:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.425+0000 7f765b7fe700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f764806ace0 0x7f764806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.426+0000 7f765b7fe700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f764806ace0 0x7f764806d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f764c000f60 tx=0x7f764c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.429+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7650054900 con 0x7f765c0b81c0 2023-07-06T21:40:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.625+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f765c04e1d0 con 0x7f765c0b81c0 2023-07-06T21:40:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.625+0000 7f7643fff700 1 -- 172.21.15.84:0/2970753002 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7650057f20 con 0x7f765c0b81c0 2023-07-06T21:40:33.629 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:33.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f764806ace0 msgr2=0x7f764806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.631 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f764806ace0 0x7f764806d190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f764c000f60 tx=0x7f764c009040 comp rx=0 tx=0).stop 2023-07-06T21:40:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 msgr2=0x7f765c157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 0x7f765c157300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f765000d800 tx=0x7f765000dbc0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 shutdown_connections 2023-07-06T21:40:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f764806ace0 0x7f764806d190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f765c0cfd10 0x7f765c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f765c0b1a80 0x7f765c148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 --2- 172.21.15.84:0/2970753002 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f765c0b81c0 0x7f765c157300 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 >> 172.21.15.84:0/2970753002 conn(0x7f765c06b2b0 msgr2=0x7f765c0d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 shutdown_connections 2023-07-06T21:40:33.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.628+0000 7f7661fff700 1 -- 172.21.15.84:0/2970753002 wait complete. 2023-07-06T21:40:33.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:33.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.842+0000 7f33177ff700 1 Processor -- start 2023-07-06T21:40:33.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.842+0000 7f33177ff700 1 -- start start 2023-07-06T21:40:33.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.843+0000 7f33177ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b1a80 0x7f33100d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.843+0000 7f33177ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100d1830 con 0x7f33100b81c0 2023-07-06T21:40:33.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.843+0000 7f33177ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100cf330 con 0x7f33100cfd10 2023-07-06T21:40:33.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.843+0000 7f33177ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33100d1270 con 0x7f33100b1a80 2023-07-06T21:40:33.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f331559b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33100b81c0 0x7f33100cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46464/0 (socket says 172.21.15.84:46464) 2023-07-06T21:40:33.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f331559b700 1 -- 172.21.15.84:0/3454562898 learned_addr learned my addr 172.21.15.84:0/3454562898 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:33.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3315d9c700 1 --2- 172.21.15.84:0/3454562898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b1a80 0x7f33100d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624141543 0 0) 0x7f33100d1830 con 0x7f33100b81c0 2023-07-06T21:40:33.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32fc0035b0 con 0x7f33100b81c0 2023-07-06T21:40:33.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3478367058 0 0) 0x7f33100cf330 con 0x7f33100cfd10 2023-07-06T21:40:33.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33100d1830 con 0x7f33100cfd10 2023-07-06T21:40:33.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 326012030 0 0) 0x7f32fc0035b0 con 0x7f33100b81c0 2023-07-06T21:40:33.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3315d9c700 1 -- 172.21.15.84:0/3454562898 >> v1:172.21.15.107:6789/0 conn(0x7f33100cfd10 legacy=0x7f33100d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3315d9c700 1 -- 172.21.15.84:0/3454562898 >> v1:172.21.15.84:6789/0 conn(0x7f33100b81c0 legacy=0x7f33100cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.844+0000 7f3315d9c700 1 -- 172.21.15.84:0/3454562898 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100d1550 con 0x7f33100b1a80 2023-07-06T21:40:33.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.845+0000 7f3315d9c700 1 --2- 172.21.15.84:0/3454562898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b1a80 0x7f33100d2a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3308009dc0 tx=0x7f330800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4136a8a45219f4e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.845+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800e070 con 0x7f33100b1a80 2023-07-06T21:40:33.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.845+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308003a20 con 0x7f33100b1a80 2023-07-06T21:40:33.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.845+0000 7f3307fff700 1 -- 172.21.15.84:0/3454562898 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c001670 con 0x7f33100b1a80 2023-07-06T21:40:33.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.845+0000 7f33177ff700 1 -- 172.21.15.84:0/3454562898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b1a80 msgr2=0x7f33100d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.846+0000 7f33177ff700 1 --2- 172.21.15.84:0/3454562898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b1a80 0x7f33100d2a90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3308009dc0 tx=0x7f330800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.846+0000 7f33177ff700 1 -- 172.21.15.84:0/3454562898 shutdown_connections 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.846+0000 7f33177ff700 1 --2- 172.21.15.84:0/3454562898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b1a80 0x7f33100d2a90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.846+0000 7f33177ff700 1 -- 172.21.15.84:0/3454562898 >> 172.21.15.84:0/3454562898 conn(0x7f331006b2b0 msgr2=0x7f331006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.846+0000 7f33177ff700 1 -- 172.21.15.84:0/3454562898 shutdown_connections 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.846+0000 7f33177ff700 1 -- 172.21.15.84:0/3454562898 wait complete. 2023-07-06T21:40:33.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.847+0000 7f33177ff700 1 Processor -- start 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.847+0000 7f33177ff700 1 -- start start 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.848+0000 7f33177ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 0x7f3310158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.848+0000 7f33177ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b81c0 0x7f3310156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.848+0000 7f33177ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33100cfd10 0x7f33101f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.848+0000 7f33177ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3310158200 con 0x7f33100b1a80 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.848+0000 7f33177ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101575f0 con 0x7f33100cfd10 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.848+0000 7f33177ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33101578d0 con 0x7f33100b81c0 2023-07-06T21:40:33.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 0x7f3310158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 0x7f3310158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52268/0 (socket says 172.21.15.84:52268) 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 -- 172.21.15.84:0/1162766045 learned_addr learned my addr 172.21.15.84:0/1162766045 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f3315d9c700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33100cfd10 0x7f33101f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f3314d9a700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b81c0 0x7f3310156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 -- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b81c0 msgr2=0x7f3310156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b81c0 0x7f3310156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 -- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33100cfd10 msgr2=0x7f33101f1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:33.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33100cfd10 0x7f33101f1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 -- 172.21.15.84:0/1162766045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3308009a70 con 0x7f33100b1a80 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f331559b700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 0x7f3310158690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c750 tx=0x7f330c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.849+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c007560 con 0x7f33100b1a80 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.850+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c0076c0 con 0x7f33100b1a80 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.850+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3310157bb0 con 0x7f33100b1a80 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.850+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c004a70 con 0x7f33100b1a80 2023-07-06T21:40:33.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.850+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3310158bd0 con 0x7f33100b1a80 2023-07-06T21:40:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.851+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33101584e0 con 0x7f33100b1a80 2023-07-06T21:40:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.852+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f330c004bd0 con 0x7f33100b1a80 2023-07-06T21:40:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.853+0000 7f3305ffb700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f32f406ace0 0x7f32f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.853+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f330c088a40 con 0x7f33100b1a80 2023-07-06T21:40:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.853+0000 7f3314d9a700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f32f406ace0 0x7f32f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:33.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.854+0000 7f3314d9a700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f32f406ace0 0x7f32f406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3300007fd0 tx=0x7f3300009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:33.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:33.857+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f330c08ae60 con 0x7f33100b1a80 2023-07-06T21:40:34.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.053+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f331004e1d0 con 0x7f33100b1a80 2023-07-06T21:40:34.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.054+0000 7f3305ffb700 1 -- 172.21.15.84:0/1162766045 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f330c0180a0 con 0x7f33100b1a80 2023-07-06T21:40:34.057 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f32f406ace0 msgr2=0x7f32f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f32f406ace0 0x7f32f406d190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3300007fd0 tx=0x7f3300009040 comp rx=0 tx=0).stop 2023-07-06T21:40:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 msgr2=0x7f3310158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 0x7f3310158690 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f330c00c750 tx=0x7f330c00cb10 comp rx=0 tx=0).stop 2023-07-06T21:40:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 shutdown_connections 2023-07-06T21:40:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f32f406ace0 0x7f32f406d190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33100cfd10 0x7f33101f1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33100b1a80 0x7f3310158690 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 --2- 172.21.15.84:0/1162766045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33100b81c0 0x7f3310156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 >> 172.21.15.84:0/1162766045 conn(0x7f331006b2b0 msgr2=0x7f33100d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 shutdown_connections 2023-07-06T21:40:34.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.056+0000 7f33177ff700 1 -- 172.21.15.84:0/1162766045 wait complete. 2023-07-06T21:40:34.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.270+0000 7f758c9ff700 1 Processor -- start 2023-07-06T21:40:34.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.271+0000 7f758c9ff700 1 -- start start 2023-07-06T21:40:34.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758c9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75880b81c0 0x7f75880cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758c9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880ceda0 0x7f75880cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758c9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75880d1830 con 0x7f75880b1a80 2023-07-06T21:40:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758c9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75880cf330 con 0x7f75880b81c0 2023-07-06T21:40:34.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758c9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75880d1270 con 0x7f75880ceda0 2023-07-06T21:40:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f7586d9d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880ceda0 0x7f75880cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758659c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75880b81c0 0x7f75880cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758659c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75880b81c0 0x7f75880cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:51914/0 (socket says 172.21.15.84:51914) 2023-07-06T21:40:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.272+0000 7f758659c700 1 -- 172.21.15.84:0/688515190 learned_addr learned my addr 172.21.15.84:0/688515190 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f758559a700 1 -- 172.21.15.84:0/688515190 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846201830 0 0) 0x7f75880d1830 con 0x7f75880b1a80 2023-07-06T21:40:34.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f758559a700 1 -- 172.21.15.84:0/688515190 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75700035b0 con 0x7f75880b1a80 2023-07-06T21:40:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f7586d9d700 1 -- 172.21.15.84:0/688515190 >> v2:172.21.15.107:3300/0 conn(0x7f75880b81c0 msgr2=0x7f75880cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f7586d9d700 1 --2- 172.21.15.84:0/688515190 >> v2:172.21.15.107:3300/0 conn(0x7f75880b81c0 0x7f75880cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f7586d9d700 1 -- 172.21.15.84:0/688515190 >> v1:172.21.15.107:6789/0 conn(0x7f75880b1a80 legacy=0x7f758815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f7586d9d700 1 -- 172.21.15.84:0/688515190 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880d1550 con 0x7f75880ceda0 2023-07-06T21:40:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.273+0000 7f7586d9d700 1 --2- 172.21.15.84:0/688515190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880ceda0 0x7f75880cfb40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7580009d90 tx=0x7f758000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12e2a7d1719a630 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758559a700 1 -- 172.21.15.84:0/688515190 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000e070 con 0x7f75880ceda0 2023-07-06T21:40:34.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758559a700 1 -- 172.21.15.84:0/688515190 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758000bb20 con 0x7f75880ceda0 2023-07-06T21:40:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758559a700 1 -- 172.21.15.84:0/688515190 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758000be90 con 0x7f75880ceda0 2023-07-06T21:40:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 -- 172.21.15.84:0/688515190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880ceda0 msgr2=0x7f75880cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 --2- 172.21.15.84:0/688515190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880ceda0 0x7f75880cfb40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7580009d90 tx=0x7f758000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 -- 172.21.15.84:0/688515190 shutdown_connections 2023-07-06T21:40:34.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 --2- 172.21.15.84:0/688515190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880ceda0 0x7f75880cfb40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 --2- 172.21.15.84:0/688515190 >> v2:172.21.15.107:3300/0 conn(0x7f75880b81c0 0x7f75880cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 -- 172.21.15.84:0/688515190 >> 172.21.15.84:0/688515190 conn(0x7f758806b2b0 msgr2=0x7f758806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 -- 172.21.15.84:0/688515190 shutdown_connections 2023-07-06T21:40:34.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.274+0000 7f758c9ff700 1 -- 172.21.15.84:0/688515190 wait complete. 2023-07-06T21:40:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.276+0000 7f758c9ff700 1 Processor -- start 2023-07-06T21:40:34.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.276+0000 7f758c9ff700 1 -- start start 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.277+0000 7f758c9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880b1a80 0x7f75880c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.277+0000 7f758c9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75880b81c0 0x7f75880cc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.277+0000 7f758c9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 0x7f75880b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f758c9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75881f26d0 con 0x7f75880ceda0 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f758c9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c95e0 con 0x7f75880b81c0 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f758c9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75881f1b10 con 0x7f75880b1a80 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 0x7f75880b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 0x7f75880b6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52282/0 (socket says 172.21.15.84:52282) 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 -- 172.21.15.84:0/902096255 learned_addr learned my addr 172.21.15.84:0/902096255 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f758659c700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880b1a80 0x7f75880c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7585d9b700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75880b81c0 0x7f75880cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 -- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880b1a80 msgr2=0x7f75880c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880b1a80 0x7f75880c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 -- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75880b81c0 msgr2=0x7f75880cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75880b81c0 0x7f75880cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.278+0000 7f7586d9d700 1 -- 172.21.15.84:0/902096255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580009a70 con 0x7f75880ceda0 2023-07-06T21:40:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.279+0000 7f7586d9d700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 0x7f75880b6a70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f75880d13b0 tx=0x7f7580004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.279+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580007000 con 0x7f75880ceda0 2023-07-06T21:40:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.279+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580012650 con 0x7f75880ceda0 2023-07-06T21:40:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.279+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580012a00 con 0x7f75880ceda0 2023-07-06T21:40:34.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.279+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75880ccd40 con 0x7f75880ceda0 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.279+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75881f1d30 con 0x7f75880ceda0 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.280+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75881f2810 con 0x7f75880ceda0 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.281+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7580012b60 con 0x7f75880ceda0 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.282+0000 7f7576ffd700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f756806ace0 0x7f756806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.282+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f758008b140 con 0x7f75880ceda0 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.282+0000 7f758659c700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f756806ace0 0x7f756806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.283+0000 7f758659c700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f756806ace0 0x7f756806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f75880cc130 tx=0x7f75780092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:34.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.287+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7580056370 con 0x7f75880ceda0 2023-07-06T21:40:34.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.481+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75880b8590 con 0x7f75880ceda0 2023-07-06T21:40:34.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.481+0000 7f7576ffd700 1 -- 172.21.15.84:0/902096255 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7580059990 con 0x7f75880ceda0 2023-07-06T21:40:34.485 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f756806ace0 msgr2=0x7f756806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f756806ace0 0x7f756806d190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f75880cc130 tx=0x7f75780092c0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 msgr2=0x7f75880b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 0x7f75880b6a70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f75880d13b0 tx=0x7f7580004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 shutdown_connections 2023-07-06T21:40:34.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f756806ace0 0x7f756806d190 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75880ceda0 0x7f75880b6a70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75880b1a80 0x7f75880c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 --2- 172.21.15.84:0/902096255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75880b81c0 0x7f75880cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 >> 172.21.15.84:0/902096255 conn(0x7f758806b2b0 msgr2=0x7f758806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:34.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 shutdown_connections 2023-07-06T21:40:34.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.484+0000 7f758c9ff700 1 -- 172.21.15.84:0/902096255 wait complete. 2023-07-06T21:40:34.505 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:40:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.696+0000 7efe20fff700 1 Processor -- start 2023-07-06T21:40:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.696+0000 7efe20fff700 1 -- start start 2023-07-06T21:40:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.697+0000 7efe20fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efe1c0b8530 0x7efe1c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.697+0000 7efe20fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efe1c0d3580 0x7efe1c0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.697+0000 7efe20fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c0d0990 con 0x7efe1c0cf3b0 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe20fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c0d3af0 con 0x7efe1c0d3580 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe20fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c0cd9d0 con 0x7efe1c0b8530 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe13fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7efe1c0d3580 0x7efe1c0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe1ad9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7efe1c0cf3b0 0x7efe1c15feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46476/0 (socket says 172.21.15.84:46476) 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe1ad9d700 1 -- 172.21.15.84:0/2970452737 learned_addr learned my addr 172.21.15.84:0/2970452737 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe1a59c700 1 --2- 172.21.15.84:0/2970452737 >> v2:172.21.15.107:3300/0 conn(0x7efe1c0b8530 0x7efe1c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe13fff700 1 -- 172.21.15.84:0/2970452737 >> v2:172.21.15.107:3300/0 conn(0x7efe1c0b8530 msgr2=0x7efe1c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe13fff700 1 --2- 172.21.15.84:0/2970452737 >> v2:172.21.15.107:3300/0 conn(0x7efe1c0b8530 0x7efe1c0cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe13fff700 1 -- 172.21.15.84:0/2970452737 >> v1:172.21.15.84:6789/0 conn(0x7efe1c0cf3b0 legacy=0x7efe1c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe13fff700 1 -- 172.21.15.84:0/2970452737 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c0cd410 con 0x7efe1c0d3580 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe19d9b700 1 -- 172.21.15.84:0/2970452737 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923733185 0 0) 0x7efe1c0d0990 con 0x7efe1c0cf3b0 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.698+0000 7efe13fff700 1 --2- 172.21.15.84:0/2970452737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0d3580 0x7efe1c0d0e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe1c06c680 tx=0x7efe04009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2174036c894a287a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe19d9b700 1 -- 172.21.15.84:0/2970452737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04011830 con 0x7efe1c0d3580 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe19d9b700 1 -- 172.21.15.84:0/2970452737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe04011990 con 0x7efe1c0d3580 2023-07-06T21:40:34.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe19d9b700 1 -- 172.21.15.84:0/2970452737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04011d20 con 0x7efe1c0d3580 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 -- 172.21.15.84:0/2970452737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0d3580 msgr2=0x7efe1c0d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 --2- 172.21.15.84:0/2970452737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0d3580 0x7efe1c0d0e70 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efe1c06c680 tx=0x7efe04009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 -- 172.21.15.84:0/2970452737 shutdown_connections 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 --2- 172.21.15.84:0/2970452737 >> v2:172.21.15.107:3300/0 conn(0x7efe1c0b8530 0x7efe1c0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 --2- 172.21.15.84:0/2970452737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0d3580 0x7efe1c0d0e70 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 -- 172.21.15.84:0/2970452737 >> 172.21.15.84:0/2970452737 conn(0x7efe1c0715c0 msgr2=0x7efe1c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:34.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 -- 172.21.15.84:0/2970452737 shutdown_connections 2023-07-06T21:40:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.699+0000 7efe20fff700 1 -- 172.21.15.84:0/2970452737 wait complete. 2023-07-06T21:40:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.701+0000 7efe20fff700 1 Processor -- start 2023-07-06T21:40:34.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.701+0000 7efe20fff700 1 -- start start 2023-07-06T21:40:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe20fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 0x7efe1c0b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe20fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0cf3b0 0x7efe1c154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe20fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0d3580 0x7efe1c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe20fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0c6750 con 0x7efe1c0b8530 2023-07-06T21:40:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe20fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0d22d0 con 0x7efe1c0cf3b0 2023-07-06T21:40:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe20fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0c69a0 con 0x7efe1c0d3580 2023-07-06T21:40:34.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe1a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 0x7efe1c0b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe1a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 0x7efe1c0b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52292/0 (socket says 172.21.15.84:52292) 2023-07-06T21:40:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.702+0000 7efe1a59c700 1 -- 172.21.15.84:0/2350087472 learned_addr learned my addr 172.21.15.84:0/2350087472 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe13fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0cf3b0 0x7efe1c154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1ad9d700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0d3580 0x7efe1c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1a59c700 1 -- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0d3580 msgr2=0x7efe1c154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1a59c700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0d3580 0x7efe1c154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1a59c700 1 -- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0cf3b0 msgr2=0x7efe1c154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1a59c700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0cf3b0 0x7efe1c154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1a59c700 1 -- 172.21.15.84:0/2350087472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040094a0 con 0x7efe1c0b8530 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe13fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0cf3b0 0x7efe1c154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe1a59c700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 0x7efe1c0b30a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00e730 tx=0x7efe0c00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c013610 con 0x7efe1c0b8530 2023-07-06T21:40:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c013770 con 0x7efe1c0b8530 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c0c6c80 con 0x7efe1c0b8530 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c013b20 con 0x7efe1c0b8530 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.703+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c0d1f60 con 0x7efe1c0b8530 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.705+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1c04e240 con 0x7efe1c0b8530 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.705+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7efe0c013c80 con 0x7efe1c0b8530 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.706+0000 7efe12ffd700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efdfc06ace0 0x7efdfc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:34.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.706+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7efe0c088f30 con 0x7efe1c0b8530 2023-07-06T21:40:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.706+0000 7efe13fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efdfc06ace0 0x7efdfc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.707+0000 7efe13fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efdfc06ace0 0x7efdfc06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe040090b0 tx=0x7efe0401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:34.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.710+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7efe0c0579a0 con 0x7efe1c0b8530 2023-07-06T21:40:34.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.921+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efe1c15ce60 con 0x7efe1c0b8530 2023-07-06T21:40:34.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.921+0000 7efe12ffd700 1 -- 172.21.15.84:0/2350087472 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7efe0c021020 con 0x7efe1c0b8530 2023-07-06T21:40:34.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efdfc06ace0 msgr2=0x7efdfc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efdfc06ace0 0x7efdfc06d190 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe040090b0 tx=0x7efe0401c040 comp rx=0 tx=0).stop 2023-07-06T21:40:34.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 msgr2=0x7efe1c0b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:34.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 0x7efe1c0b30a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00e730 tx=0x7efe0c00eaf0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 shutdown_connections 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7efdfc06ace0 0x7efdfc06d190 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0d3580 0x7efe1c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe1c0b8530 0x7efe1c0b30a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 --2- 172.21.15.84:0/2350087472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0cf3b0 0x7efe1c154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 >> 172.21.15.84:0/2350087472 conn(0x7efe1c0715c0 msgr2=0x7efe1c15fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:34.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 shutdown_connections 2023-07-06T21:40:34.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:34.924+0000 7efe20fff700 1 -- 172.21.15.84:0/2350087472 wait complete. 2023-07-06T21:40:34.929 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:40:34.945 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2023-07-06T21:40:34.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:34.945 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-07-06T21:40:35.020 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:35.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:40:35.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:40:35.022 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T21:40:35.066 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T21:40:35.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:40:35.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.256+0000 7fa95afff700 1 Processor -- start 2023-07-06T21:40:35.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.257+0000 7fa95afff700 1 -- start start 2023-07-06T21:40:35.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa95afff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9540d3710 0x7fa9540cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa95afff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa9540cfa90 0x7fa9540d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa95afff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9540cd910 con 0x7fa9540b83c0 2023-07-06T21:40:35.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa95afff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9540cff10 con 0x7fa9540d3710 2023-07-06T21:40:35.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa95afff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9540d0a80 con 0x7fa9540cfa90 2023-07-06T21:40:35.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa958d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa9540b83c0 0x7fa9540cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46492/0 (socket says 172.21.15.84:46492) 2023-07-06T21:40:35.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa953fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa9540d3710 0x7fa9540cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa958d9b700 1 -- 172.21.15.84:0/2087579411 learned_addr learned my addr 172.21.15.84:0/2087579411 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:35.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa95959c700 1 --2- 172.21.15.84:0/2087579411 >> v2:172.21.15.107:3300/0 conn(0x7fa9540cfa90 0x7fa9540d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa953fff700 1 -- 172.21.15.84:0/2087579411 >> v2:172.21.15.107:3300/0 conn(0x7fa9540cfa90 msgr2=0x7fa9540d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:35.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa953fff700 1 --2- 172.21.15.84:0/2087579411 >> v2:172.21.15.107:3300/0 conn(0x7fa9540cfa90 0x7fa9540d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.258+0000 7fa953fff700 1 -- 172.21.15.84:0/2087579411 >> v1:172.21.15.84:6789/0 conn(0x7fa9540b83c0 legacy=0x7fa9540cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa953fff700 1 -- 172.21.15.84:0/2087579411 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540b8790 con 0x7fa9540d3710 2023-07-06T21:40:35.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa9537fe700 1 -- 172.21.15.84:0/2087579411 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1831704012 0 0) 0x7fa948001110 con 0x7fa9540b83c0 2023-07-06T21:40:35.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa953fff700 1 --2- 172.21.15.84:0/2087579411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540d3710 0x7fa9540cf550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa9540b1b00 tx=0x7fa940009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e2e146fadcc67f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:35.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa9537fe700 1 -- 172.21.15.84:0/2087579411 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940011830 con 0x7fa9540d3710 2023-07-06T21:40:35.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa9537fe700 1 -- 172.21.15.84:0/2087579411 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa940011990 con 0x7fa9540d3710 2023-07-06T21:40:35.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa9537fe700 1 -- 172.21.15.84:0/2087579411 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940011d20 con 0x7fa9540d3710 2023-07-06T21:40:35.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa95afff700 1 -- 172.21.15.84:0/2087579411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540d3710 msgr2=0x7fa9540cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.259+0000 7fa95afff700 1 --2- 172.21.15.84:0/2087579411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540d3710 0x7fa9540cf550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa9540b1b00 tx=0x7fa940009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:35.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.260+0000 7fa95afff700 1 -- 172.21.15.84:0/2087579411 shutdown_connections 2023-07-06T21:40:35.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.260+0000 7fa95afff700 1 --2- 172.21.15.84:0/2087579411 >> v2:172.21.15.107:3300/0 conn(0x7fa9540cfa90 0x7fa9540d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.260+0000 7fa95afff700 1 --2- 172.21.15.84:0/2087579411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540d3710 0x7fa9540cf550 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.260+0000 7fa95afff700 1 -- 172.21.15.84:0/2087579411 >> 172.21.15.84:0/2087579411 conn(0x7fa95406b350 msgr2=0x7fa95406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:35.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.260+0000 7fa95afff700 1 -- 172.21.15.84:0/2087579411 shutdown_connections 2023-07-06T21:40:35.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.260+0000 7fa95afff700 1 -- 172.21.15.84:0/2087579411 wait complete. 2023-07-06T21:40:35.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.261+0000 7fa95afff700 1 Processor -- start 2023-07-06T21:40:35.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.261+0000 7fa95afff700 1 -- start start 2023-07-06T21:40:35.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.262+0000 7fa95afff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9540b83c0 0x7fa954153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.262+0000 7fa95afff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 0x7fa9540b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.262+0000 7fa95afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9540d3710 0x7fa9540b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.262+0000 7fa95afff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9541f3370 con 0x7fa9540cfa90 2023-07-06T21:40:35.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.262+0000 7fa95afff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95415e610 con 0x7fa9540b83c0 2023-07-06T21:40:35.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.262+0000 7fa95afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9541f3090 con 0x7fa9540d3710 2023-07-06T21:40:35.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 0x7fa9540b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 0x7fa9540b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52304/0 (socket says 172.21.15.84:52304) 2023-07-06T21:40:35.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 -- 172.21.15.84:0/4032755650 learned_addr learned my addr 172.21.15.84:0/4032755650 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:35.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa958d9b700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9540b83c0 0x7fa954153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa95959c700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9540d3710 0x7fa9540b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 -- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9540d3710 msgr2=0x7fa9540b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9540d3710 0x7fa9540b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 -- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9540b83c0 msgr2=0x7fa954153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9540b83c0 0x7fa954153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 -- 172.21.15.84:0/4032755650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400094a0 con 0x7fa9540cfa90 2023-07-06T21:40:35.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa95959c700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9540d3710 0x7fa9540b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:35.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa953fff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 0x7fa9540b69c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa940013000 tx=0x7fa940004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:35.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.263+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9400036a0 con 0x7fa9540cfa90 2023-07-06T21:40:35.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.264+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94001f070 con 0x7fa9540cfa90 2023-07-06T21:40:35.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.264+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940003a20 con 0x7fa9540cfa90 2023-07-06T21:40:35.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.264+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9540cb7d0 con 0x7fa9540cfa90 2023-07-06T21:40:35.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.264+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9540cbd90 con 0x7fa9540cfa90 2023-07-06T21:40:35.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.265+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9540cb630 con 0x7fa9540cfa90 2023-07-06T21:40:35.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.265+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa940003b80 con 0x7fa9540cfa90 2023-07-06T21:40:35.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.267+0000 7fa9517fa700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa93806ace0 0x7fa93806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.267+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa940021070 con 0x7fa9540cfa90 2023-07-06T21:40:35.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.267+0000 7fa958d9b700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa93806ace0 0x7fa93806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.267+0000 7fa958d9b700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa93806ace0 0x7fa93806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa948000f60 tx=0x7fa948007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:35.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.271+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9540cb630 con 0x7fa9540cfa90 2023-07-06T21:40:35.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.510+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9540cb630 con 0x7fa9540cfa90 2023-07-06T21:40:35.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.510+0000 7fa9517fa700 1 -- 172.21.15.84:0/4032755650 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7fa9540cb630 con 0x7fa9540cfa90 2023-07-06T21:40:35.514 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:35.514 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:40:35.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa93806ace0 msgr2=0x7fa93806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa93806ace0 0x7fa93806d190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa948000f60 tx=0x7fa948007e70 comp rx=0 tx=0).stop 2023-07-06T21:40:35.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 msgr2=0x7fa9540b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 0x7fa9540b69c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa940013000 tx=0x7fa940004750 comp rx=0 tx=0).stop 2023-07-06T21:40:35.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 shutdown_connections 2023-07-06T21:40:35.517 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa93806ace0 0x7fa93806d190 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9540d3710 0x7fa9540b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9540b83c0 0x7fa954153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 --2- 172.21.15.84:0/4032755650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9540cfa90 0x7fa9540b69c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 >> 172.21.15.84:0/4032755650 conn(0x7fa95406b350 msgr2=0x7fa9540d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:35.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 shutdown_connections 2023-07-06T21:40:35.518 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.513+0000 7fa95afff700 1 -- 172.21.15.84:0/4032755650 wait complete. 2023-07-06T21:40:35.519 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:40:35.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:35.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.726+0000 7f700adff700 1 Processor -- start 2023-07-06T21:40:35.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.726+0000 7f700adff700 1 -- start start 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.727+0000 7f700adff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 0x7f70040cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.727+0000 7f700adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040cfc60 0x7f70040d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.727+0000 7f700adff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040d0a80 con 0x7f70040d3710 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.727+0000 7f700adff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70040b8790 con 0x7f70040b83c0 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.727+0000 7f700adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040b4e70 con 0x7f70040cfc60 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f7008b9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 0x7f70040cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f700939c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040cfc60 0x7f70040d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f7008b9b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 0x7f70040cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:51962/0 (socket says 172.21.15.84:51962) 2023-07-06T21:40:35.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f7008b9b700 1 -- 172.21.15.84:0/3679195378 learned_addr learned my addr 172.21.15.84:0/3679195378 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:35.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f700939c700 1 -- 172.21.15.84:0/3679195378 >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 msgr2=0x7f70040cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:35.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f700939c700 1 --2- 172.21.15.84:0/3679195378 >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 0x7f70040cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f700939c700 1 -- 172.21.15.84:0/3679195378 >> v1:172.21.15.107:6789/0 conn(0x7f70040d3710 legacy=0x7f70040cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f700939c700 1 -- 172.21.15.84:0/3679195378 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040b5150 con 0x7f70040cfc60 2023-07-06T21:40:35.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f7008b9b700 1 --2- 172.21.15.84:0/3679195378 >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 0x7f70040cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.728+0000 7f700939c700 1 --2- 172.21.15.84:0/3679195378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040cfc60 0x7f70040d0f10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa5ca0ada501f697 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f70037fe700 1 -- 172.21.15.84:0/3679195378 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00e070 con 0x7f70040cfc60 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f70037fe700 1 -- 172.21.15.84:0/3679195378 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00bb20 con 0x7f70040cfc60 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f70037fe700 1 -- 172.21.15.84:0/3679195378 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc012430 con 0x7f70040cfc60 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 -- 172.21.15.84:0/3679195378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040cfc60 msgr2=0x7f70040d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 --2- 172.21.15.84:0/3679195378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040cfc60 0x7f70040d0f10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009d90 tx=0x7f6ffc00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:35.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 -- 172.21.15.84:0/3679195378 shutdown_connections 2023-07-06T21:40:35.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 --2- 172.21.15.84:0/3679195378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040cfc60 0x7f70040d0f10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 --2- 172.21.15.84:0/3679195378 >> v2:172.21.15.107:3300/0 conn(0x7f70040b83c0 0x7f70040cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 -- 172.21.15.84:0/3679195378 >> 172.21.15.84:0/3679195378 conn(0x7f700406b350 msgr2=0x7f700406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:35.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 -- 172.21.15.84:0/3679195378 shutdown_connections 2023-07-06T21:40:35.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.729+0000 7f700adff700 1 -- 172.21.15.84:0/3679195378 wait complete. 2023-07-06T21:40:35.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.731+0000 7f700adff700 1 Processor -- start 2023-07-06T21:40:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.731+0000 7f700adff700 1 -- start start 2023-07-06T21:40:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f700adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040b83c0 0x7f700414a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f700adff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 0x7f700415db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f700adff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70040d3710 0x7f70041f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f700adff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700415e600 con 0x7f70040cfc60 2023-07-06T21:40:35.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f700adff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700415e8e0 con 0x7f70040d3710 2023-07-06T21:40:35.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f700adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700415ebc0 con 0x7f70040b83c0 2023-07-06T21:40:35.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f7003fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 0x7f700415db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f7003fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 0x7f700415db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52310/0 (socket says 172.21.15.84:52310) 2023-07-06T21:40:35.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f7003fff700 1 -- 172.21.15.84:0/4106860362 learned_addr learned my addr 172.21.15.84:0/4106860362 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:35.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.732+0000 7f7008b9b700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040b83c0 0x7f700414a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f700939c700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70040d3710 0x7f70041f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f7003fff700 1 -- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040b83c0 msgr2=0x7f700414a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f7003fff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040b83c0 0x7f700414a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f7003fff700 1 -- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70040d3710 msgr2=0x7f70041f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f7003fff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70040d3710 0x7f70041f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f7003fff700 1 -- 172.21.15.84:0/4106860362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8009550 con 0x7f70040cfc60 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f7003fff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 0x7f700415db00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c4b0 tx=0x7f6ff000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00052d0 con 0x7f70040cfc60 2023-07-06T21:40:35.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0005430 con 0x7f70040cfc60 2023-07-06T21:40:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00038c0 con 0x7f70040cfc60 2023-07-06T21:40:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc009a70 con 0x7f70040cfc60 2023-07-06T21:40:35.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.733+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700415bd40 con 0x7f70040cfc60 2023-07-06T21:40:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.734+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700415bf50 con 0x7f70040cfc60 2023-07-06T21:40:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.735+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6ff0003a20 con 0x7f70040cfc60 2023-07-06T21:40:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.736+0000 7f70017fa700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fe806ace0 0x7f6fe806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.737+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6ff0088850 con 0x7f70040cfc60 2023-07-06T21:40:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.737+0000 7f7008b9b700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fe806ace0 0x7f6fe806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:35.741 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.737+0000 7f7008b9b700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fe806ace0 0x7f6fe806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8006d30 tx=0x7f6ff80092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:35.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.740+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6ff00538c0 con 0x7f70040cfc60 2023-07-06T21:40:35.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.938+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70040cdb80 con 0x7f70040cfc60 2023-07-06T21:40:35.942 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.939+0000 7f70017fa700 1 -- 172.21.15.84:0/4106860362 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f6ff0056ee0 con 0x7f70040cfc60 2023-07-06T21:40:35.943 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:35.943 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fe806ace0 msgr2=0x7f6fe806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fe806ace0 0x7f6fe806d190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8006d30 tx=0x7f6ff80092c0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 msgr2=0x7f700415db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:35.945 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 0x7f700415db00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000c4b0 tx=0x7f6ff000c7c0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 shutdown_connections 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6fe806ace0 0x7f6fe806d190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70040d3710 0x7f70041f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70040b83c0 0x7f700414a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 --2- 172.21.15.84:0/4106860362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70040cfc60 0x7f700415db00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 >> 172.21.15.84:0/4106860362 conn(0x7f700406b350 msgr2=0x7f700415fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:35.946 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 shutdown_connections 2023-07-06T21:40:35.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:35.942+0000 7f700adff700 1 -- 172.21.15.84:0/4106860362 wait complete. 2023-07-06T21:40:35.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:39:24.496506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:39:25.404949+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:35.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:36.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.158+0000 7f879bdff700 1 Processor -- start 2023-07-06T21:40:36.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.159+0000 7f879bdff700 1 -- start start 2023-07-06T21:40:36.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879bdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b81c0 0x7f87940cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879bdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87940b1a80 0x7f87940cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879bdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87940d1830 con 0x7f87940ceda0 2023-07-06T21:40:36.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879bdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87940cf330 con 0x7f87940b1a80 2023-07-06T21:40:36.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879bdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87940d1270 con 0x7f87940b81c0 2023-07-06T21:40:36.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879939a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87940b1a80 0x7f87940cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879939a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f87940b1a80 0x7f87940cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52322/0 (socket says 172.21.15.84:52322) 2023-07-06T21:40:36.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f879939a700 1 -- 172.21.15.84:0/2268375023 learned_addr learned my addr 172.21.15.84:0/2268375023 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:36.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.160+0000 7f8799b9b700 1 --2- 172.21.15.84:0/2268375023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b81c0 0x7f87940cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f879939a700 1 -- 172.21.15.84:0/2268375023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b81c0 msgr2=0x7f87940cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:36.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f879939a700 1 --2- 172.21.15.84:0/2268375023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b81c0 0x7f87940cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f879939a700 1 -- 172.21.15.84:0/2268375023 >> v1:172.21.15.107:6789/0 conn(0x7f87940ceda0 legacy=0x7f879415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f879939a700 1 -- 172.21.15.84:0/2268375023 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87940d1550 con 0x7f87940b1a80 2023-07-06T21:40:36.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f879939a700 1 --2- 172.21.15.84:0/2268375023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940b1a80 0x7f87940cfb40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f87840098f0 tx=0x7f8784009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2268b6478970db server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:36.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f8798b99700 1 -- 172.21.15.84:0/2268375023 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784011830 con 0x7f87940b1a80 2023-07-06T21:40:36.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f8798b99700 1 -- 172.21.15.84:0/2268375023 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784011990 con 0x7f87940b1a80 2023-07-06T21:40:36.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.161+0000 7f8798b99700 1 -- 172.21.15.84:0/2268375023 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8784011d20 con 0x7f87940b1a80 2023-07-06T21:40:36.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 -- 172.21.15.84:0/2268375023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940b1a80 msgr2=0x7f87940cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 --2- 172.21.15.84:0/2268375023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940b1a80 0x7f87940cfb40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f87840098f0 tx=0x7f8784009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:36.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 -- 172.21.15.84:0/2268375023 shutdown_connections 2023-07-06T21:40:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 --2- 172.21.15.84:0/2268375023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b81c0 0x7f87940cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 --2- 172.21.15.84:0/2268375023 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940b1a80 0x7f87940cfb40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 -- 172.21.15.84:0/2268375023 >> 172.21.15.84:0/2268375023 conn(0x7f879406b2b0 msgr2=0x7f879406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 -- 172.21.15.84:0/2268375023 shutdown_connections 2023-07-06T21:40:36.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.162+0000 7f879bdff700 1 -- 172.21.15.84:0/2268375023 wait complete. 2023-07-06T21:40:36.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.164+0000 7f879bdff700 1 Processor -- start 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.165+0000 7f879bdff700 1 -- start start 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879bdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b1a80 0x7f8794159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879bdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87940b81c0 0x7f8794153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879bdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 0x7f87940c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879bdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87941f5d30 con 0x7f87940ceda0 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879bdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87941f50b0 con 0x7f87940b81c0 2023-07-06T21:40:36.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879bdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87941f5390 con 0x7f87940b1a80 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879a39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 0x7f87940c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879a39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 0x7f87940c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52338/0 (socket says 172.21.15.84:52338) 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879a39c700 1 -- 172.21.15.84:0/1244357210 learned_addr learned my addr 172.21.15.84:0/1244357210 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f8799b9b700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b1a80 0x7f8794159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879939a700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87940b81c0 0x7f8794153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.166+0000 7f879a39c700 1 -- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b1a80 msgr2=0x7f8794159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879a39c700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b1a80 0x7f8794159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879a39c700 1 -- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87940b81c0 msgr2=0x7f8794153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879a39c700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87940b81c0 0x7f8794153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879a39c700 1 -- 172.21.15.84:0/1244357210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784009550 con 0x7f87940ceda0 2023-07-06T21:40:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879939a700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87940b81c0 0x7f8794153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:36.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879a39c700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 0x7f87940c88f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f878c00cb50 tx=0x7f878c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:36.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c003c00 con 0x7f87940ceda0 2023-07-06T21:40:36.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c003d60 con 0x7f87940ceda0 2023-07-06T21:40:36.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c00aac0 con 0x7f87940ceda0 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87941f5670 con 0x7f87940ceda0 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.167+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879415a1c0 con 0x7f87940ceda0 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.168+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879404e1d0 con 0x7f87940ceda0 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.169+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f878c00ac20 con 0x7f87940ceda0 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.170+0000 7f878a7fc700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f877806ace0 0x7f877806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.170+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f878c089420 con 0x7f87940ceda0 2023-07-06T21:40:36.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.170+0000 7f8799b9b700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f877806ace0 0x7f877806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.171+0000 7f8799b9b700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f877806ace0 0x7f877806d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8790009be0 tx=0x7f8790009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:36.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.174+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f878c054520 con 0x7f87940ceda0 2023-07-06T21:40:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.370+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87940b4f30 con 0x7f87940ceda0 2023-07-06T21:40:36.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.370+0000 7f878a7fc700 1 -- 172.21.15.84:0/1244357210 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f878c057b40 con 0x7f87940ceda0 2023-07-06T21:40:36.374 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.372+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f877806ace0 msgr2=0x7f877806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.372+0000 7f879bdff700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f877806ace0 0x7f877806d190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8790009be0 tx=0x7f8790009210 comp rx=0 tx=0).stop 2023-07-06T21:40:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.372+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 msgr2=0x7f87940c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.372+0000 7f879bdff700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 0x7f87940c88f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f878c00cb50 tx=0x7f878c00ce60 comp rx=0 tx=0).stop 2023-07-06T21:40:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 shutdown_connections 2023-07-06T21:40:36.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f877806ace0 0x7f877806d190 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f87940ceda0 0x7f87940c88f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87940b1a80 0x7f8794159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 --2- 172.21.15.84:0/1244357210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87940b81c0 0x7f8794153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 >> 172.21.15.84:0/1244357210 conn(0x7f879406b2b0 msgr2=0x7f87940d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 shutdown_connections 2023-07-06T21:40:36.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.373+0000 7f879bdff700 1 -- 172.21.15.84:0/1244357210 wait complete. 2023-07-06T21:40:36.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:36.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.587+0000 7f65e31ff700 1 Processor -- start 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.587+0000 7f65e31ff700 1 -- start start 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e31ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f65dc0b1a80 0x7f65dc0d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e31ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0d1830 con 0x7f65dc0b81c0 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e31ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65dc0cf330 con 0x7f65dc0cfd10 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e31ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc0d1270 con 0x7f65dc0b1a80 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e0f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f65dc0b81c0 0x7f65dc0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46502/0 (socket says 172.21.15.84:46502) 2023-07-06T21:40:36.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e0f9b700 1 -- 172.21.15.84:0/4130441897 learned_addr learned my addr 172.21.15.84:0/4130441897 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:36.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e179c700 1 --2- 172.21.15.84:0/4130441897 >> v2:172.21.15.84:3300/0 conn(0x7f65dc0b1a80 0x7f65dc0d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65db7fe700 1 -- 172.21.15.84:0/4130441897 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193810641 0 0) 0x7f65dc0d1830 con 0x7f65dc0b81c0 2023-07-06T21:40:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65db7fe700 1 -- 172.21.15.84:0/4130441897 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65cc0035b0 con 0x7f65dc0b81c0 2023-07-06T21:40:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e179c700 1 -- 172.21.15.84:0/4130441897 >> v1:172.21.15.107:6789/0 conn(0x7f65dc0cfd10 legacy=0x7f65dc0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e179c700 1 -- 172.21.15.84:0/4130441897 >> v1:172.21.15.84:6789/0 conn(0x7f65dc0b81c0 legacy=0x7f65dc0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.589+0000 7f65e179c700 1 -- 172.21.15.84:0/4130441897 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc0d1550 con 0x7f65dc0b1a80 2023-07-06T21:40:36.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.590+0000 7f65e179c700 1 --2- 172.21.15.84:0/4130441897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0b1a80 0x7f65dc0d2a90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009d90 tx=0x7f65d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36fe57960c8eafe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:36.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.590+0000 7f65db7fe700 1 -- 172.21.15.84:0/4130441897 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400e070 con 0x7f65dc0b1a80 2023-07-06T21:40:36.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.590+0000 7f65db7fe700 1 -- 172.21.15.84:0/4130441897 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d400bb20 con 0x7f65dc0b1a80 2023-07-06T21:40:36.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.590+0000 7f65db7fe700 1 -- 172.21.15.84:0/4130441897 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d400be90 con 0x7f65dc0b1a80 2023-07-06T21:40:36.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.590+0000 7f65e31ff700 1 -- 172.21.15.84:0/4130441897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0b1a80 msgr2=0x7f65dc0d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.590+0000 7f65e31ff700 1 --2- 172.21.15.84:0/4130441897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0b1a80 0x7f65dc0d2a90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009d90 tx=0x7f65d400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:36.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.591+0000 7f65e31ff700 1 -- 172.21.15.84:0/4130441897 shutdown_connections 2023-07-06T21:40:36.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.591+0000 7f65e31ff700 1 --2- 172.21.15.84:0/4130441897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0b1a80 0x7f65dc0d2a90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.591+0000 7f65e31ff700 1 -- 172.21.15.84:0/4130441897 >> 172.21.15.84:0/4130441897 conn(0x7f65dc06b2b0 msgr2=0x7f65dc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:36.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.591+0000 7f65e31ff700 1 -- 172.21.15.84:0/4130441897 shutdown_connections 2023-07-06T21:40:36.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.591+0000 7f65e31ff700 1 -- 172.21.15.84:0/4130441897 wait complete. 2023-07-06T21:40:36.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.592+0000 7f65e31ff700 1 Processor -- start 2023-07-06T21:40:36.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.592+0000 7f65e31ff700 1 -- start start 2023-07-06T21:40:36.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.593+0000 7f65e31ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65dc0b1a80 0x7f65dc156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.593+0000 7f65e31ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65dc0b81c0 0x7f65dc1f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.593+0000 7f65e31ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 0x7f65dc0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.593+0000 7f65e31ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc15d690 con 0x7f65dc0cfd10 2023-07-06T21:40:36.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.593+0000 7f65e31ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc15cac0 con 0x7f65dc0b1a80 2023-07-06T21:40:36.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.593+0000 7f65e31ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc15cd60 con 0x7f65dc0b81c0 2023-07-06T21:40:36.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 0x7f65dc0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 0x7f65dc0c6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52358/0 (socket says 172.21.15.84:52358) 2023-07-06T21:40:36.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 -- 172.21.15.84:0/1534778687 learned_addr learned my addr 172.21.15.84:0/1534778687 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:36.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e0f9b700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65dc0b1a80 0x7f65dc156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65dbfff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65dc0b81c0 0x7f65dc1f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 -- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65dc0b81c0 msgr2=0x7f65dc1f6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65dc0b81c0 0x7f65dc1f6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 -- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65dc0b1a80 msgr2=0x7f65dc156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65dc0b1a80 0x7f65dc156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 -- 172.21.15.84:0/1534778687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d4009a70 con 0x7f65dc0cfd10 2023-07-06T21:40:36.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.594+0000 7f65e179c700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 0x7f65dc0c6ec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b370 tx=0x7f65d4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:36.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.595+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d4004960 con 0x7f65dc0cfd10 2023-07-06T21:40:36.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.595+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d4020070 con 0x7f65dc0cfd10 2023-07-06T21:40:36.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.595+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65dc15d040 con 0x7f65dc0cfd10 2023-07-06T21:40:36.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.595+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d40276c0 con 0x7f65dc0cfd10 2023-07-06T21:40:36.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.595+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc1f6150 con 0x7f65dc0cfd10 2023-07-06T21:40:36.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.596+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc0666c0 con 0x7f65dc0cfd10 2023-07-06T21:40:36.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.597+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f65d4012430 con 0x7f65dc0cfd10 2023-07-06T21:40:36.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.598+0000 7f65d97fa700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c006ace0 0x7f65c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:36.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.598+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f65d4022070 con 0x7f65dc0cfd10 2023-07-06T21:40:36.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.598+0000 7f65e0f9b700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c006ace0 0x7f65c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:36.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.598+0000 7f65e0f9b700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c006ace0 0x7f65c006d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d00011e0 tx=0x7f65d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:36.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.602+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f65d4056580 con 0x7f65dc0cfd10 2023-07-06T21:40:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.802+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65dc1f62e0 con 0x7f65dc0cfd10 2023-07-06T21:40:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.803+0000 7f65d97fa700 1 -- 172.21.15.84:0/1534778687 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65d4059ba0 con 0x7f65dc0cfd10 2023-07-06T21:40:36.807 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:36.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c006ace0 msgr2=0x7f65c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c006ace0 0x7f65c006d190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d00011e0 tx=0x7f65d0009040 comp rx=0 tx=0).stop 2023-07-06T21:40:36.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 msgr2=0x7f65dc0c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:36.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 0x7f65dc0c6ec0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b370 tx=0x7f65d4004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 shutdown_connections 2023-07-06T21:40:36.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f65c006ace0 0x7f65c006d190 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65dc0cfd10 0x7f65dc0c6ec0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65dc0b1a80 0x7f65dc156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 --2- 172.21.15.84:0/1534778687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f65dc0b81c0 0x7f65dc1f6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:36.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 >> 172.21.15.84:0/1534778687 conn(0x7f65dc06b2b0 msgr2=0x7f65dc0cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:36.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 shutdown_connections 2023-07-06T21:40:36.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:36.805+0000 7f65e31ff700 1 -- 172.21.15.84:0/1534778687 wait complete. 2023-07-06T21:40:36.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:40:37.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.019+0000 7f83049ff700 1 Processor -- start 2023-07-06T21:40:37.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.019+0000 7f83049ff700 1 -- start start 2023-07-06T21:40:37.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.020+0000 7f83049ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83000b81c0 0x7f83000cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.020+0000 7f83049ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83000b1a80 0x7f83000cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.020+0000 7f83049ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83000d1830 con 0x7f83000ceda0 2023-07-06T21:40:37.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.020+0000 7f83049ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000cf330 con 0x7f83000b81c0 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.020+0000 7f83049ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83000d1270 con 0x7f83000b1a80 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83000b81c0 0x7f83000cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83000b81c0 0x7f83000cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52362/0 (socket says 172.21.15.84:52362) 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 -- 172.21.15.84:0/668088771 learned_addr learned my addr 172.21.15.84:0/668088771 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fdd9b700 1 --2- 172.21.15.84:0/668088771 >> v2:172.21.15.107:3300/0 conn(0x7f83000b1a80 0x7f83000cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 -- 172.21.15.84:0/668088771 >> v2:172.21.15.107:3300/0 conn(0x7f83000b1a80 msgr2=0x7f83000cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 --2- 172.21.15.84:0/668088771 >> v2:172.21.15.107:3300/0 conn(0x7f83000b1a80 0x7f83000cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 -- 172.21.15.84:0/668088771 >> v1:172.21.15.107:6789/0 conn(0x7f83000ceda0 legacy=0x7f830015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 -- 172.21.15.84:0/668088771 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83000d1550 con 0x7f83000b81c0 2023-07-06T21:40:37.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.021+0000 7f82fe59c700 1 --2- 172.21.15.84:0/668088771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f83000cf600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83000d0770 tx=0x7f82f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1253705ffe9ff033 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f82fd59a700 1 -- 172.21.15.84:0/668088771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000b870 con 0x7f83000b81c0 2023-07-06T21:40:37.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f82fd59a700 1 -- 172.21.15.84:0/668088771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f000b9d0 con 0x7f83000b81c0 2023-07-06T21:40:37.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f82fd59a700 1 -- 172.21.15.84:0/668088771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f000bd60 con 0x7f83000b81c0 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 -- 172.21.15.84:0/668088771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 msgr2=0x7f83000cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 --2- 172.21.15.84:0/668088771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f83000cf600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83000d0770 tx=0x7f82f0009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 -- 172.21.15.84:0/668088771 shutdown_connections 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 --2- 172.21.15.84:0/668088771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f83000cf600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 --2- 172.21.15.84:0/668088771 >> v2:172.21.15.107:3300/0 conn(0x7f83000b1a80 0x7f83000cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 -- 172.21.15.84:0/668088771 >> 172.21.15.84:0/668088771 conn(0x7f830006b2b0 msgr2=0x7f830006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:37.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 -- 172.21.15.84:0/668088771 shutdown_connections 2023-07-06T21:40:37.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.022+0000 7f83049ff700 1 -- 172.21.15.84:0/668088771 wait complete. 2023-07-06T21:40:37.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.025+0000 7f83049ff700 1 Processor -- start 2023-07-06T21:40:37.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.025+0000 7f83049ff700 1 -- start start 2023-07-06T21:40:37.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f83049ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83000b1a80 0x7f8300155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f83049ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f8300159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f83049ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83000ceda0 0x7f830015a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f83049ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c6a90 con 0x7f83000b81c0 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f83049ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c6d70 con 0x7f83000b1a80 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f83049ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83000c7050 con 0x7f83000ceda0 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f82fdd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f8300159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f82fdd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f8300159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52370/0 (socket says 172.21.15.84:52370) 2023-07-06T21:40:37.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f82fdd9b700 1 -- 172.21.15.84:0/1047680523 learned_addr learned my addr 172.21.15.84:0/1047680523 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f82fe59c700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83000b1a80 0x7f8300155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.026+0000 7f82fed9d700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83000ceda0 0x7f830015a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82fdd9b700 1 -- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83000ceda0 msgr2=0x7f830015a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82fdd9b700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83000ceda0 0x7f830015a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82fdd9b700 1 -- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83000b1a80 msgr2=0x7f8300155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82fdd9b700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83000b1a80 0x7f8300155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82fdd9b700 1 -- 172.21.15.84:0/1047680523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0009550 con 0x7f83000b81c0 2023-07-06T21:40:37.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82fdd9b700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f8300159820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f82f400d800 tx=0x7f82f400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40125f0 con 0x7f83000b81c0 2023-07-06T21:40:37.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4012750 con 0x7f83000b81c0 2023-07-06T21:40:37.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83000c7330 con 0x7f83000b81c0 2023-07-06T21:40:37.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4012b00 con 0x7f83000b81c0 2023-07-06T21:40:37.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.027+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8300156220 con 0x7f83000b81c0 2023-07-06T21:40:37.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.029+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83000c7850 con 0x7f83000b81c0 2023-07-06T21:40:37.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.029+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f82f4012c60 con 0x7f83000b81c0 2023-07-06T21:40:37.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.030+0000 7f82eeffd700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f82e006ace0 0x7f82e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.030+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f82f4089500 con 0x7f83000b81c0 2023-07-06T21:40:37.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.031+0000 7f82fe59c700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f82e006ace0 0x7f82e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.031+0000 7f82fe59c700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f82e006ace0 0x7f82e006d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82f0000c00 tx=0x7f82f0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.035+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83000c7850 con 0x7f83000b81c0 2023-07-06T21:40:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.234+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83000c7850 con 0x7f83000b81c0 2023-07-06T21:40:37.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.234+0000 7f82eeffd700 1 -- 172.21.15.84:0/1047680523 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83000c7850 con 0x7f83000b81c0 2023-07-06T21:40:37.237 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:40:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.236+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f82e006ace0 msgr2=0x7f82e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.236+0000 7f83049ff700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f82e006ace0 0x7f82e006d190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f82f0000c00 tx=0x7f82f0008030 comp rx=0 tx=0).stop 2023-07-06T21:40:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.236+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 msgr2=0x7f8300159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.236+0000 7f83049ff700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f8300159820 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f82f400d800 tx=0x7f82f400db10 comp rx=0 tx=0).stop 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.236+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 shutdown_connections 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f82e006ace0 0x7f82e006d190 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83000ceda0 0x7f830015a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83000b1a80 0x7f8300155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 --2- 172.21.15.84:0/1047680523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83000b81c0 0x7f8300159820 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 >> 172.21.15.84:0/1047680523 conn(0x7f830006b2b0 msgr2=0x7f83000cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:37.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 shutdown_connections 2023-07-06T21:40:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.237+0000 7f83049ff700 1 -- 172.21.15.84:0/1047680523 wait complete. 2023-07-06T21:40:37.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:40:37.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.450+0000 7fc451fff700 1 Processor -- start 2023-07-06T21:40:37.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.451+0000 7fc451fff700 1 -- start start 2023-07-06T21:40:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc451fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc451fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc44c0ceda0 0x7fc44c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc451fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c0d1830 con 0x7fc44c0b1a80 2023-07-06T21:40:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc451fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0cf330 con 0x7fc44c0ceda0 2023-07-06T21:40:37.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc451fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0d1270 con 0x7fc44c0b81c0 2023-07-06T21:40:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc44c0ceda0 0x7fc44c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc44c0ceda0 0x7fc44c0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52378/0 (socket says 172.21.15.84:52378) 2023-07-06T21:40:37.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 -- 172.21.15.84:0/464555676 learned_addr learned my addr 172.21.15.84:0/464555676 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:37.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 -- 172.21.15.84:0/464555676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 msgr2=0x7fc44c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:37.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 --2- 172.21.15.84:0/464555676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 -- 172.21.15.84:0/464555676 >> v1:172.21.15.107:6789/0 conn(0x7fc44c0b1a80 legacy=0x7fc44c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.452+0000 7fc44bfff700 1 -- 172.21.15.84:0/464555676 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0d1550 con 0x7fc44c0ceda0 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc44bfff700 1 --2- 172.21.15.84:0/464555676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0cfb40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc444009e50 tx=0x7fc44400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff719088f3ef52c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc44affd700 1 -- 172.21.15.84:0/464555676 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400e070 con 0x7fc44c0ceda0 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc44affd700 1 -- 172.21.15.84:0/464555676 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444003a20 con 0x7fc44c0ceda0 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc44affd700 1 -- 172.21.15.84:0/464555676 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444003dd0 con 0x7fc44c0ceda0 2023-07-06T21:40:37.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc451fff700 1 -- 172.21.15.84:0/464555676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 msgr2=0x7fc44c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc451fff700 1 --2- 172.21.15.84:0/464555676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0cfb40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc444009e50 tx=0x7fc44400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc451fff700 1 -- 172.21.15.84:0/464555676 shutdown_connections 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc451fff700 1 --2- 172.21.15.84:0/464555676 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0cfb40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc451fff700 1 --2- 172.21.15.84:0/464555676 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.453+0000 7fc451fff700 1 -- 172.21.15.84:0/464555676 >> 172.21.15.84:0/464555676 conn(0x7fc44c06b2b0 msgr2=0x7fc44c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.454+0000 7fc451fff700 1 -- 172.21.15.84:0/464555676 shutdown_connections 2023-07-06T21:40:37.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.454+0000 7fc451fff700 1 -- 172.21.15.84:0/464555676 wait complete. 2023-07-06T21:40:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.455+0000 7fc451fff700 1 Processor -- start 2023-07-06T21:40:37.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.455+0000 7fc451fff700 1 -- start start 2023-07-06T21:40:37.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.456+0000 7fc451fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc44c0b1a80 0x7fc44c0c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.456+0000 7fc451fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c1561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.456+0000 7fc451fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.456+0000 7fc451fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0ca200 con 0x7fc44c0ceda0 2023-07-06T21:40:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.456+0000 7fc451fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0ca3f0 con 0x7fc44c0b1a80 2023-07-06T21:40:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.456+0000 7fc451fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c0ca6d0 con 0x7fc44c0b81c0 2023-07-06T21:40:37.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52390/0 (socket says 172.21.15.84:52390) 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 -- 172.21.15.84:0/1043475509 learned_addr learned my addr 172.21.15.84:0/1043475509 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44b7fe700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc44c0b1a80 0x7fc44c0c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc443fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c1561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 -- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 msgr2=0x7fc44c1561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c1561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 -- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc44c0b1a80 msgr2=0x7fc44c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc44c0b1a80 0x7fc44c0c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 -- 172.21.15.84:0/1043475509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009b30 con 0x7fc44c0ceda0 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44b7fe700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc44c0b1a80 0x7fc44c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc44bfff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0c9cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc444015040 tx=0x7fc444004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444004440 con 0x7fc44c0ceda0 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4440045c0 con 0x7fc44c0ceda0 2023-07-06T21:40:37.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.457+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c0ca9b0 con 0x7fc44c0ceda0 2023-07-06T21:40:37.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.458+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444004970 con 0x7fc44c0ceda0 2023-07-06T21:40:37.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.458+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c0caf70 con 0x7fc44c0ceda0 2023-07-06T21:40:37.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.459+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44c0d0d20 con 0x7fc44c0ceda0 2023-07-06T21:40:37.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.460+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc444012430 con 0x7fc44c0ceda0 2023-07-06T21:40:37.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.461+0000 7fc448ff9700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc42c06ace0 0x7fc42c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.461+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc44408aa50 con 0x7fc44c0ceda0 2023-07-06T21:40:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.461+0000 7fc44b7fe700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc42c06ace0 0x7fc42c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.462+0000 7fc44b7fe700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc42c06ace0 0x7fc42c06d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0099d0 tx=0x7fc43c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.465+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc444055ad0 con 0x7fc44c0ceda0 2023-07-06T21:40:37.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.659+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc44c04e1d0 con 0x7fc44c0ceda0 2023-07-06T21:40:37.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.660+0000 7fc448ff9700 1 -- 172.21.15.84:0/1043475509 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4440590f0 con 0x7fc44c0ceda0 2023-07-06T21:40:37.663 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:40:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc42c06ace0 msgr2=0x7fc42c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc42c06ace0 0x7fc42c06d190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0099d0 tx=0x7fc43c008040 comp rx=0 tx=0).stop 2023-07-06T21:40:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 msgr2=0x7fc44c0c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0c9cc0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc444015040 tx=0x7fc444004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 shutdown_connections 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc42c06ace0 0x7fc42c06d190 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc44c0ceda0 0x7fc44c0c9cc0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc44c0b1a80 0x7fc44c0c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 --2- 172.21.15.84:0/1043475509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc44c0b81c0 0x7fc44c1561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 >> 172.21.15.84:0/1043475509 conn(0x7fc44c06b2b0 msgr2=0x7fc44c0d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:37.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 shutdown_connections 2023-07-06T21:40:37.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.662+0000 7fc451fff700 1 -- 172.21.15.84:0/1043475509 wait complete. 2023-07-06T21:40:37.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:40:37.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.871+0000 7f2041bff700 1 Processor -- start 2023-07-06T21:40:37.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.871+0000 7f2041bff700 1 -- start start 2023-07-06T21:40:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.872+0000 7f2041bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c0d2690 0x7f203c0cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f2041bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c0d1460 con 0x7f203c0b1fd0 2023-07-06T21:40:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f2041bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c0d1740 con 0x7f203c0b59c0 2023-07-06T21:40:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f2041bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0d2530 con 0x7f203c0d2690 2023-07-06T21:40:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f203b7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f203c0b1fd0 0x7f203c0cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46514/0 (socket says 172.21.15.84:46514) 2023-07-06T21:40:37.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f203b7fe700 1 -- 172.21.15.84:0/4223313589 learned_addr learned my addr 172.21.15.84:0/4223313589 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f203affd700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c0d2690 0x7f203c0cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078133464 0 0) 0x7f203c0d1460 con 0x7f203c0b1fd0 2023-07-06T21:40:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.873+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20240035b0 con 0x7f203c0b1fd0 2023-07-06T21:40:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2118763760 0 0) 0x7f20240035b0 con 0x7f203c0b1fd0 2023-07-06T21:40:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c0d2690 msgr2=0x7f203c0cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:37.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c0d2690 0x7f203c0cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 >> v1:172.21.15.107:6789/0 conn(0x7f203c0b59c0 legacy=0x7f203c15fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0cd260 con 0x7f203c0b1fd0 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f202c001670 con 0x7f203c0b1fd0 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 >> v1:172.21.15.84:6789/0 conn(0x7f203c0b1fd0 legacy=0x7f203c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2024003a30 0x7f2024003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240043d0 0x7f2024004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f202400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0d2530 con 0x7f20240050e0 2023-07-06T21:40:37.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202400c000 con 0x7f2024003a30 2023-07-06T21:40:37.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202400c1c0 con 0x7f20240043d0 2023-07-06T21:40:37.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203affd700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c0d2690 0x7f203c0cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:37.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203bfff700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f202400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203b7fe700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2024003a30 0x7f2024003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203affd700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240043d0 0x7f2024004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203bfff700 1 -- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240043d0 msgr2=0x7f2024004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203bfff700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f20240043d0 0x7f2024004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203bfff700 1 -- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2024003a30 msgr2=0x7f2024003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203bfff700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2024003a30 0x7f2024003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.874+0000 7f203bfff700 1 -- 172.21.15.84:0/4223313589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20240035b0 con 0x7f20240050e0 2023-07-06T21:40:37.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f203bfff700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f202400ba10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f203400c290 tx=0x7f2024005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b4d7ed5ac8c259 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034001e70 con 0x7f20240050e0 2023-07-06T21:40:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203400ce50 con 0x7f20240050e0 2023-07-06T21:40:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f203a7fc700 1 -- 172.21.15.84:0/4223313589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034012660 con 0x7f20240050e0 2023-07-06T21:40:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f2041bff700 1 -- 172.21.15.84:0/4223313589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 msgr2=0x7f202400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f2041bff700 1 --2- 172.21.15.84:0/4223313589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f202400ba10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f203400c290 tx=0x7f2024005880 comp rx=0 tx=0).stop 2023-07-06T21:40:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.875+0000 7f203bfff700 1 -- 172.21.15.84:0/4223313589 reap_dead start 2023-07-06T21:40:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.876+0000 7f2041bff700 1 -- 172.21.15.84:0/4223313589 shutdown_connections 2023-07-06T21:40:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.876+0000 7f2041bff700 1 -- 172.21.15.84:0/4223313589 >> 172.21.15.84:0/4223313589 conn(0x7f203c06b340 msgr2=0x7f203c06b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.876+0000 7f2041bff700 1 -- 172.21.15.84:0/4223313589 shutdown_connections 2023-07-06T21:40:37.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.876+0000 7f2041bff700 1 -- 172.21.15.84:0/4223313589 wait complete. 2023-07-06T21:40:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.878+0000 7f2041bff700 1 Processor -- start 2023-07-06T21:40:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.878+0000 7f2041bff700 1 -- start start 2023-07-06T21:40:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.879+0000 7f2041bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f203c14e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.879+0000 7f2041bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c14b7b0 0x7f203c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.879+0000 7f2041bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f203c14ae60 0x7f203c0b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f2041bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c14cba0 con 0x7f20240050e0 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f2041bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c14bf90 con 0x7f203c14ae60 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f2041bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c14c270 con 0x7f203c14b7b0 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f203c14e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f203c14e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52410/0 (socket says 172.21.15.84:52410) 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 -- 172.21.15.84:0/3475952859 learned_addr learned my addr 172.21.15.84:0/3475952859 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 -- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c14b7b0 msgr2=0x7f203c14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203bfff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f203c14ae60 0x7f203c0b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203affd700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c14b7b0 0x7f203c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c14b7b0 0x7f203c14d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 -- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f203c14ae60 msgr2=0x7f203c0b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f203c14ae60 0x7f203c0b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 -- 172.21.15.84:0/3475952859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c14c550 con 0x7f20240050e0 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203bfff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f203c14ae60 0x7f203c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.880+0000 7f203b7fe700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f203c14e3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f202c0017a0 tx=0x7f202c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.881+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00f6c0 con 0x7f20240050e0 2023-07-06T21:40:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.881+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00f840 con 0x7f20240050e0 2023-07-06T21:40:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.881+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c14c830 con 0x7f20240050e0 2023-07-06T21:40:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.881+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00fbf0 con 0x7f20240050e0 2023-07-06T21:40:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.881+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c14bc20 con 0x7f20240050e0 2023-07-06T21:40:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.882+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c14ce80 con 0x7f20240050e0 2023-07-06T21:40:37.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.883+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f202c00fd50 con 0x7f20240050e0 2023-07-06T21:40:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.883+0000 7f2023fff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f202806ace0 0x7f202806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.883+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f202c089510 con 0x7f20240050e0 2023-07-06T21:40:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.884+0000 7f203affd700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f202806ace0 0x7f202806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:37.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.884+0000 7f203affd700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f202806ace0 0x7f202806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f203c1e5fc0 tx=0x7f2030008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:37.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:37.887+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f202c054580 con 0x7f20240050e0 2023-07-06T21:40:38.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.128+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f203c0d1110 con 0x7f20240050e0 2023-07-06T21:40:38.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.128+0000 7f2023fff700 1 -- 172.21.15.84:0/3475952859 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f202c057ba0 con 0x7f20240050e0 2023-07-06T21:40:38.132 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:38.132 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:40:38.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f202806ace0 msgr2=0x7f202806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f202806ace0 0x7f202806d190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f203c1e5fc0 tx=0x7f2030008040 comp rx=0 tx=0).stop 2023-07-06T21:40:38.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 msgr2=0x7f203c14e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f203c14e3c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f202c0017a0 tx=0x7f202c007620 comp rx=0 tx=0).stop 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 shutdown_connections 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f202806ace0 0x7f202806d190 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f203c14ae60 0x7f203c0b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20240050e0 0x7f203c14e3c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 --2- 172.21.15.84:0/3475952859 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f203c14b7b0 0x7f203c14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 >> 172.21.15.84:0/3475952859 conn(0x7f203c06b340 msgr2=0x7f203c15f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:38.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 shutdown_connections 2023-07-06T21:40:38.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.131+0000 7f2041bff700 1 -- 172.21.15.84:0/3475952859 wait complete. 2023-07-06T21:40:38.136 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:40:38.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:38.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.345+0000 7f52897ff700 1 Processor -- start 2023-07-06T21:40:38.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.345+0000 7f52897ff700 1 -- start start 2023-07-06T21:40:38.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.346+0000 7f52897ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f52840b1fd0 0x7f52840cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.346+0000 7f52897ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f52840d2690 0x7f52840cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.346+0000 7f52897ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52840d1460 con 0x7f52840b59c0 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.346+0000 7f52897ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52840d2530 con 0x7f52840d2690 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.346+0000 7f52897ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52840d1740 con 0x7f52840b1fd0 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.346+0000 7f52827fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f52840d2690 0x7f52840cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f52840d2690 0x7f52840cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52424/0 (socket says 172.21.15.84:52424) 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 -- 172.21.15.84:0/434780886 learned_addr learned my addr 172.21.15.84:0/434780886 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f5282ffd700 1 --2- 172.21.15.84:0/434780886 >> v2:172.21.15.107:3300/0 conn(0x7f52840b1fd0 0x7f52840cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 -- 172.21.15.84:0/434780886 >> v2:172.21.15.107:3300/0 conn(0x7f52840b1fd0 msgr2=0x7f52840cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:38.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 --2- 172.21.15.84:0/434780886 >> v2:172.21.15.107:3300/0 conn(0x7f52840b1fd0 0x7f52840cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 -- 172.21.15.84:0/434780886 >> v1:172.21.15.107:6789/0 conn(0x7f52840b59c0 legacy=0x7f52840d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:38.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 -- 172.21.15.84:0/434780886 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52840cd260 con 0x7f52840d2690 2023-07-06T21:40:38.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f52827fc700 1 --2- 172.21.15.84:0/434780886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840d2690 0x7f52840cf6a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f528408bce0 tx=0x7f5278009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2057f2c65ad9e39 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:38.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f5281ffb700 1 -- 172.21.15.84:0/434780886 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5278004230 con 0x7f52840d2690 2023-07-06T21:40:38.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f5281ffb700 1 -- 172.21.15.84:0/434780886 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5278004390 con 0x7f52840d2690 2023-07-06T21:40:38.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.347+0000 7f5281ffb700 1 -- 172.21.15.84:0/434780886 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52780036a0 con 0x7f52840d2690 2023-07-06T21:40:38.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 -- 172.21.15.84:0/434780886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840d2690 msgr2=0x7f52840cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 --2- 172.21.15.84:0/434780886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840d2690 0x7f52840cf6a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f528408bce0 tx=0x7f5278009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 -- 172.21.15.84:0/434780886 shutdown_connections 2023-07-06T21:40:38.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 --2- 172.21.15.84:0/434780886 >> v2:172.21.15.107:3300/0 conn(0x7f52840b1fd0 0x7f52840cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 --2- 172.21.15.84:0/434780886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840d2690 0x7f52840cf6a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 -- 172.21.15.84:0/434780886 >> 172.21.15.84:0/434780886 conn(0x7f528406b340 msgr2=0x7f528406b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:38.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 -- 172.21.15.84:0/434780886 shutdown_connections 2023-07-06T21:40:38.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.348+0000 7f52897ff700 1 -- 172.21.15.84:0/434780886 wait complete. 2023-07-06T21:40:38.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.350+0000 7f52897ff700 1 Processor -- start 2023-07-06T21:40:38.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.350+0000 7f52897ff700 1 -- start start 2023-07-06T21:40:38.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52897ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 0x7f52840c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52897ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52840b59c0 0x7f5284149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52897ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52840d2690 0x7f528414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52897ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52840b86b0 con 0x7f52840b1fd0 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52897ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528414b080 con 0x7f52840d2690 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52897ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528414a1d0 con 0x7f52840b59c0 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f5282ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 0x7f52840c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f5282ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 0x7f52840c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52436/0 (socket says 172.21.15.84:52436) 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f5282ffd700 1 -- 172.21.15.84:0/1347337316 learned_addr learned my addr 172.21.15.84:0/1347337316 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52827fc700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52840b59c0 0x7f5284149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f52837fe700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52840d2690 0x7f528414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.351+0000 7f5282ffd700 1 -- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52840b59c0 msgr2=0x7f5284149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f5282ffd700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52840b59c0 0x7f5284149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f5282ffd700 1 -- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52840d2690 msgr2=0x7f528414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f5282ffd700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52840d2690 0x7f528414b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f5282ffd700 1 -- 172.21.15.84:0/1347337316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5278009550 con 0x7f52840b1fd0 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f52827fc700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52840b59c0 0x7f5284149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:38.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f5282ffd700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 0x7f52840c8a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f527400e810 tx=0x7f527400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:38.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274013610 con 0x7f52840b1fd0 2023-07-06T21:40:38.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274013770 con 0x7f52840b1fd0 2023-07-06T21:40:38.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528414a470 con 0x7f52840b1fd0 2023-07-06T21:40:38.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274013b20 con 0x7f52840b1fd0 2023-07-06T21:40:38.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.352+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528414aa30 con 0x7f52840b1fd0 2023-07-06T21:40:38.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.353+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528404e240 con 0x7f52840b1fd0 2023-07-06T21:40:38.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.354+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5274013c80 con 0x7f52840b1fd0 2023-07-06T21:40:38.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.355+0000 7f526b7fe700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f527006ace0 0x7f527006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.355+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f52740892b0 con 0x7f52840b1fd0 2023-07-06T21:40:38.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.355+0000 7f52827fc700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f527006ace0 0x7f527006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.356+0000 7f52827fc700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f527006ace0 0x7f527006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f528408bce0 tx=0x7f5278008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:38.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.359+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5274054290 con 0x7f52840b1fd0 2023-07-06T21:40:38.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.561+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52840b5f30 con 0x7f52840b1fd0 2023-07-06T21:40:38.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.562+0000 7f526b7fe700 1 -- 172.21.15.84:0/1347337316 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7f52740578b0 con 0x7f52840b1fd0 2023-07-06T21:40:38.565 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:38.565 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:38.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f527006ace0 msgr2=0x7f527006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f527006ace0 0x7f527006d190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f528408bce0 tx=0x7f5278008030 comp rx=0 tx=0).stop 2023-07-06T21:40:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 msgr2=0x7f52840c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 0x7f52840c8a70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f527400e810 tx=0x7f527400ebd0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 shutdown_connections 2023-07-06T21:40:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f527006ace0 0x7f527006d190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52840d2690 0x7f528414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52840b1fd0 0x7f52840c8a70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 --2- 172.21.15.84:0/1347337316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52840b59c0 0x7f5284149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.564+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 >> 172.21.15.84:0/1347337316 conn(0x7f528406b340 msgr2=0x7f52840b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:38.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.565+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 shutdown_connections 2023-07-06T21:40:38.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.565+0000 7f52897ff700 1 -- 172.21.15.84:0/1347337316 wait complete. 2023-07-06T21:40:38.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T21:40:38.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:40:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.777+0000 7fbfea3ff700 1 Processor -- start 2023-07-06T21:40:38.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.777+0000 7fbfea3ff700 1 -- start start 2023-07-06T21:40:38.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.778+0000 7fbfea3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbfe40706b0 0x7fbfe40ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.778+0000 7fbfea3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe415d930 con 0x7fbfe4070230 2023-07-06T21:40:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.778+0000 7fbfea3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe415cd20 con 0x7fbfe4159130 2023-07-06T21:40:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.778+0000 7fbfea3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe415d000 con 0x7fbfe40706b0 2023-07-06T21:40:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe37fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbfe4070230 0x7fbfe40c9c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46516/0 (socket says 172.21.15.84:46516) 2023-07-06T21:40:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe37fe700 1 -- 172.21.15.84:0/732759473 learned_addr learned my addr 172.21.15.84:0/732759473 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:38.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe899c700 1 --2- 172.21.15.84:0/732759473 >> v2:172.21.15.107:3300/0 conn(0x7fbfe40706b0 0x7fbfe40ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622944256 0 0) 0x7fbfe415d930 con 0x7fbfe4070230 2023-07-06T21:40:38.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfcc0035b0 con 0x7fbfe4070230 2023-07-06T21:40:38.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615922046 0 0) 0x7fbfe415cd20 con 0x7fbfe4159130 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe415d930 con 0x7fbfe4159130 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2587568944 0 0) 0x7fbfcc0035b0 con 0x7fbfe4070230 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe899c700 1 -- 172.21.15.84:0/732759473 >> v1:172.21.15.107:6789/0 conn(0x7fbfe4159130 legacy=0x7fbfe40c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe899c700 1 -- 172.21.15.84:0/732759473 >> v1:172.21.15.84:6789/0 conn(0x7fbfe4070230 legacy=0x7fbfe40c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.779+0000 7fbfe899c700 1 -- 172.21.15.84:0/732759473 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe415d2e0 con 0x7fbfe40706b0 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.780+0000 7fbfe899c700 1 --2- 172.21.15.84:0/732759473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe40706b0 0x7fbfe40ca320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009d90 tx=0x7fbfdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1197359e7625ef03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:38.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.780+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00e070 con 0x7fbfe40706b0 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.780+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc003a20 con 0x7fbfe40706b0 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.780+0000 7fbfe2ffd700 1 -- 172.21.15.84:0/732759473 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd8001670 con 0x7fbfe40706b0 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.780+0000 7fbfea3ff700 1 -- 172.21.15.84:0/732759473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe40706b0 msgr2=0x7fbfe40ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.780+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/732759473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe40706b0 0x7fbfe40ca320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc009d90 tx=0x7fbfdc00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.781+0000 7fbfea3ff700 1 -- 172.21.15.84:0/732759473 shutdown_connections 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.781+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/732759473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe40706b0 0x7fbfe40ca320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.781+0000 7fbfea3ff700 1 -- 172.21.15.84:0/732759473 >> 172.21.15.84:0/732759473 conn(0x7fbfe40cf3d0 msgr2=0x7fbfe40cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:38.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.781+0000 7fbfea3ff700 1 -- 172.21.15.84:0/732759473 shutdown_connections 2023-07-06T21:40:38.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.781+0000 7fbfea3ff700 1 -- 172.21.15.84:0/732759473 wait complete. 2023-07-06T21:40:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.782+0000 7fbfea3ff700 1 Processor -- start 2023-07-06T21:40:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.783+0000 7fbfea3ff700 1 -- start start 2023-07-06T21:40:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.783+0000 7fbfea3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe4070230 0x7fbfe415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.783+0000 7fbfea3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfe40706b0 0x7fbfe414cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.783+0000 7fbfea3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 0x7fbfe40b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfea3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe415c050 con 0x7fbfe4159130 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfea3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe415b440 con 0x7fbfe4070230 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfea3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe415b720 con 0x7fbfe40706b0 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 0x7fbfe40b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 0x7fbfe40b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52452/0 (socket says 172.21.15.84:52452) 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 -- 172.21.15.84:0/282554162 learned_addr learned my addr 172.21.15.84:0/282554162 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe3fff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe4070230 0x7fbfe415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe37fe700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfe40706b0 0x7fbfe414cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 -- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfe40706b0 msgr2=0x7fbfe414cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfe40706b0 0x7fbfe414cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 -- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe4070230 msgr2=0x7fbfe415c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe4070230 0x7fbfe415c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 -- 172.21.15.84:0/282554162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfdc009a70 con 0x7fbfe4159130 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.784+0000 7fbfe899c700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 0x7fbfe40b2d10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b370 tx=0x7fbfdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:38.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.785+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc004440 con 0x7fbfe4159130 2023-07-06T21:40:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.785+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfdc0045a0 con 0x7fbfe4159130 2023-07-06T21:40:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.785+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc004950 con 0x7fbfe4159130 2023-07-06T21:40:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.785+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe415ba00 con 0x7fbfe4159130 2023-07-06T21:40:38.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.785+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe414d460 con 0x7fbfe4159130 2023-07-06T21:40:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.786+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe40c5190 con 0x7fbfe4159130 2023-07-06T21:40:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.787+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfdc01d470 con 0x7fbfe4159130 2023-07-06T21:40:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.788+0000 7fbfe0ff9700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfd006ad10 0x7fbfd006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.788+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fbfdc08b000 con 0x7fbfe4159130 2023-07-06T21:40:38.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.788+0000 7fbfe3fff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfd006ad10 0x7fbfd006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:38.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.789+0000 7fbfe3fff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfd006ad10 0x7fbfd006d1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40011e0 tx=0x7fbfd4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:38.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:38.792+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbfdc059b90 con 0x7fbfe4159130 2023-07-06T21:40:39.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.032+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbfe40ad820 con 0x7fbfe4159130 2023-07-06T21:40:39.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.032+0000 7fbfe0ff9700 1 -- 172.21.15.84:0/282554162 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7fbfdc059720 con 0x7fbfe4159130 2023-07-06T21:40:39.035 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:39.036 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:40:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfd006ad10 msgr2=0x7fbfd006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfd006ad10 0x7fbfd006d1c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbfd40011e0 tx=0x7fbfd4009040 comp rx=0 tx=0).stop 2023-07-06T21:40:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 msgr2=0x7fbfe40b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 0x7fbfe40b2d10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00b370 tx=0x7fbfdc004ad0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 shutdown_connections 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfd006ad10 0x7fbfd006d1c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfe4159130 0x7fbfe40b2d10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfe4070230 0x7fbfe415c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 --2- 172.21.15.84:0/282554162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfe40706b0 0x7fbfe414cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 >> 172.21.15.84:0/282554162 conn(0x7fbfe40cf3d0 msgr2=0x7fbfe40cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 shutdown_connections 2023-07-06T21:40:39.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.035+0000 7fbfea3ff700 1 -- 172.21.15.84:0/282554162 wait complete. 2023-07-06T21:40:39.040 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:40:39.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:40:39.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.247+0000 7f99585ff700 1 Processor -- start 2023-07-06T21:40:39.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.247+0000 7f99585ff700 1 -- start start 2023-07-06T21:40:39.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f99585ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99500b83c0 0x7f99500cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f99585ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99500cfc60 0x7f99500d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f99585ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99500d0a80 con 0x7f99500d3710 2023-07-06T21:40:39.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f99585ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500b8790 con 0x7f99500b83c0 2023-07-06T21:40:39.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f99585ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99500b4e70 con 0x7f99500cfc60 2023-07-06T21:40:39.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f9955b9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f99500d3710 0x7f99500cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46520/0 (socket says 172.21.15.84:46520) 2023-07-06T21:40:39.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f995639b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99500b83c0 0x7f99500cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.248+0000 7f9955b9a700 1 -- 172.21.15.84:0/919065795 learned_addr learned my addr 172.21.15.84:0/919065795 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:39.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f9956b9c700 1 --2- 172.21.15.84:0/919065795 >> v2:172.21.15.107:3300/0 conn(0x7f99500cfc60 0x7f99500d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f9955399700 1 -- 172.21.15.84:0/919065795 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296738829 0 0) 0x7f99500d0a80 con 0x7f99500d3710 2023-07-06T21:40:39.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f995639b700 1 -- 172.21.15.84:0/919065795 >> v2:172.21.15.107:3300/0 conn(0x7f99500cfc60 msgr2=0x7f99500d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:39.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f995639b700 1 --2- 172.21.15.84:0/919065795 >> v2:172.21.15.107:3300/0 conn(0x7f99500cfc60 0x7f99500d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f995639b700 1 -- 172.21.15.84:0/919065795 >> v1:172.21.15.84:6789/0 conn(0x7f99500d3710 legacy=0x7f99500cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f995639b700 1 -- 172.21.15.84:0/919065795 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99500b5150 con 0x7f99500b83c0 2023-07-06T21:40:39.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f995639b700 1 --2- 172.21.15.84:0/919065795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500b83c0 0x7f99500cd200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99500d0030 tx=0x7f994c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c141db9c909e60 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f9955399700 1 -- 172.21.15.84:0/919065795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c011830 con 0x7f99500b83c0 2023-07-06T21:40:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f9955399700 1 -- 172.21.15.84:0/919065795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c011990 con 0x7f99500b83c0 2023-07-06T21:40:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.249+0000 7f9955399700 1 -- 172.21.15.84:0/919065795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c011d20 con 0x7f99500b83c0 2023-07-06T21:40:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 -- 172.21.15.84:0/919065795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500b83c0 msgr2=0x7f99500cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 --2- 172.21.15.84:0/919065795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500b83c0 0x7f99500cd200 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f99500d0030 tx=0x7f994c009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:39.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 -- 172.21.15.84:0/919065795 shutdown_connections 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 --2- 172.21.15.84:0/919065795 >> v2:172.21.15.107:3300/0 conn(0x7f99500cfc60 0x7f99500d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 --2- 172.21.15.84:0/919065795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500b83c0 0x7f99500cd200 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 -- 172.21.15.84:0/919065795 >> 172.21.15.84:0/919065795 conn(0x7f995006b350 msgr2=0x7f995006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 -- 172.21.15.84:0/919065795 shutdown_connections 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.250+0000 7f99585ff700 1 -- 172.21.15.84:0/919065795 wait complete. 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.252+0000 7f99585ff700 1 Processor -- start 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.252+0000 7f99585ff700 1 -- start start 2023-07-06T21:40:39.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f99585ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99500b83c0 0x7f9950153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f99585ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99500cfc60 0x7f995015dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f99585ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 0x7f99500b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f99585ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015ca30 con 0x7f99500d3710 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f99585ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015cd10 con 0x7f99500cfc60 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f99585ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995015cff0 con 0x7f99500b83c0 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 0x7f99500b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 0x7f99500b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52470/0 (socket says 172.21.15.84:52470) 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 -- 172.21.15.84:0/1895354882 learned_addr learned my addr 172.21.15.84:0/1895354882 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9955b9a700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99500cfc60 0x7f995015dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f995639b700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99500b83c0 0x7f9950153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 -- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99500b83c0 msgr2=0x7f9950153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99500b83c0 0x7f9950153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 -- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99500cfc60 msgr2=0x7f995015dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99500cfc60 0x7f995015dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 -- 172.21.15.84:0/1895354882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c0094a0 con 0x7f99500d3710 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f995639b700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99500b83c0 0x7f9950153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.253+0000 7f9956b9c700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 0x7f99500b5970 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f994800ef90 tx=0x7f994800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.254+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948009b20 con 0x7f99500d3710 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.254+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994800f040 con 0x7f99500d3710 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.254+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995015d2d0 con 0x7f99500d3710 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.254+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948009e50 con 0x7f99500d3710 2023-07-06T21:40:39.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.254+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995015e010 con 0x7f99500d3710 2023-07-06T21:40:39.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.255+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f995015d7f0 con 0x7f99500d3710 2023-07-06T21:40:39.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.256+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9948014480 con 0x7f99500d3710 2023-07-06T21:40:39.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.257+0000 7f9946ffd700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f993406ace0 0x7f993406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.257+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f994808a030 con 0x7f99500d3710 2023-07-06T21:40:39.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.257+0000 7f995639b700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f993406ace0 0x7f993406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.258+0000 7f995639b700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f993406ace0 0x7f993406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f994c009b60 tx=0x7f994c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:39.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.261+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f995015d7f0 con 0x7f99500d3710 2023-07-06T21:40:39.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.500+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f995015d7f0 con 0x7f99500d3710 2023-07-06T21:40:39.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.501+0000 7f9946ffd700 1 -- 172.21.15.84:0/1895354882 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4671 (secure 0 0 0) 0x7f995015d7f0 con 0x7f99500d3710 2023-07-06T21:40:39.504 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:39.505 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/3570442738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":3570442738},{"type":"v1","addr":"172.21.15.84:6835","nonce":3570442738}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4262,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6833/2564724276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2564724276},{"type":"v1","addr":"172.21.15.107:6833","nonce":2564724276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4294,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/2710923605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":2710923605},{"type":"v1","addr":"172.21.15.84:6837","nonce":2710923605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:39:26.427365+0000","modified":"2023-07-06T21:39:29.915595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6835/2897792334","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2897792334},{"type":"v1","addr":"172.21.15.107:6835","nonce":2897792334}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-06T21:40:39.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f993406ace0 msgr2=0x7f993406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f993406ace0 0x7f993406d190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f994c009b60 tx=0x7f994c008030 comp rx=0 tx=0).stop 2023-07-06T21:40:39.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 msgr2=0x7f99500b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 0x7f99500b5970 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f994800ef90 tx=0x7f994800c3a0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 shutdown_connections 2023-07-06T21:40:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f993406ace0 0x7f993406d190 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99500d3710 0x7f99500b5970 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99500b83c0 0x7f9950153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 --2- 172.21.15.84:0/1895354882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99500cfc60 0x7f995015dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 >> 172.21.15.84:0/1895354882 conn(0x7f995006b350 msgr2=0x7f99500b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:39.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 shutdown_connections 2023-07-06T21:40:39.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.503+0000 7f99585ff700 1 -- 172.21.15.84:0/1895354882 wait complete. 2023-07-06T21:40:39.509 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2023-07-06T21:40:39.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:39.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.714+0000 7fc8693ff700 1 Processor -- start 2023-07-06T21:40:39.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.714+0000 7fc8693ff700 1 -- start start 2023-07-06T21:40:39.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.715+0000 7fc8693ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc864159130 0x7fc8640c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.715+0000 7fc8693ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8640c5000 0x7fc864070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.715+0000 7fc8693ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86415d930 con 0x7fc864070770 2023-07-06T21:40:39.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.715+0000 7fc8693ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc864070b70 con 0x7fc8640c5000 2023-07-06T21:40:39.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.715+0000 7fc8693ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86415cd20 con 0x7fc864159130 2023-07-06T21:40:39.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8640c5000 0x7fc864070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8640c5000 0x7fc864070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52484/0 (socket says 172.21.15.84:52484) 2023-07-06T21:40:39.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 -- 172.21.15.84:0/69164367 learned_addr learned my addr 172.21.15.84:0/69164367 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc862ffd700 1 --2- 172.21.15.84:0/69164367 >> v2:172.21.15.107:3300/0 conn(0x7fc864159130 0x7fc8640c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 -- 172.21.15.84:0/69164367 >> v2:172.21.15.107:3300/0 conn(0x7fc864159130 msgr2=0x7fc8640c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 --2- 172.21.15.84:0/69164367 >> v2:172.21.15.107:3300/0 conn(0x7fc864159130 0x7fc8640c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 -- 172.21.15.84:0/69164367 >> v1:172.21.15.107:6789/0 conn(0x7fc864070770 legacy=0x7fc8640cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 -- 172.21.15.84:0/69164367 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86415d000 con 0x7fc8640c5000 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.716+0000 7fc8627fc700 1 --2- 172.21.15.84:0/69164367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8640c5000 0x7fc864070230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc864067040 tx=0x7fc85800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42000e289b441b81 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:39.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc861ffb700 1 -- 172.21.15.84:0/69164367 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85800dd30 con 0x7fc8640c5000 2023-07-06T21:40:39.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc861ffb700 1 -- 172.21.15.84:0/69164367 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85800de90 con 0x7fc8640c5000 2023-07-06T21:40:39.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc861ffb700 1 -- 172.21.15.84:0/69164367 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858010690 con 0x7fc8640c5000 2023-07-06T21:40:39.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 -- 172.21.15.84:0/69164367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8640c5000 msgr2=0x7fc864070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 --2- 172.21.15.84:0/69164367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8640c5000 0x7fc864070230 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc864067040 tx=0x7fc85800be70 comp rx=0 tx=0).stop 2023-07-06T21:40:39.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 -- 172.21.15.84:0/69164367 shutdown_connections 2023-07-06T21:40:39.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 --2- 172.21.15.84:0/69164367 >> v2:172.21.15.107:3300/0 conn(0x7fc864159130 0x7fc8640c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 --2- 172.21.15.84:0/69164367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8640c5000 0x7fc864070230 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 -- 172.21.15.84:0/69164367 >> 172.21.15.84:0/69164367 conn(0x7fc8640cf3d0 msgr2=0x7fc8640cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:39.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 -- 172.21.15.84:0/69164367 shutdown_connections 2023-07-06T21:40:39.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.717+0000 7fc8693ff700 1 -- 172.21.15.84:0/69164367 wait complete. 2023-07-06T21:40:39.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.719+0000 7fc8693ff700 1 Processor -- start 2023-07-06T21:40:39.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.719+0000 7fc8693ff700 1 -- start start 2023-07-06T21:40:39.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8693ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 0x7fc8640c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8693ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8640c5000 0x7fc86415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8693ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc864159130 0x7fc8640af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8693ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640c7340 con 0x7fc864070770 2023-07-06T21:40:39.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8693ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640c7ca0 con 0x7fc864159130 2023-07-06T21:40:39.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8693ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86415a470 con 0x7fc8640c5000 2023-07-06T21:40:39.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc862ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 0x7fc8640c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc862ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 0x7fc8640c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52492/0 (socket says 172.21.15.84:52492) 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc862ffd700 1 -- 172.21.15.84:0/119926670 learned_addr learned my addr 172.21.15.84:0/119926670 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8627fc700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8640c5000 0x7fc86415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.720+0000 7fc8637fe700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc864159130 0x7fc8640af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc862ffd700 1 -- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8640c5000 msgr2=0x7fc86415bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc862ffd700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8640c5000 0x7fc86415bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc862ffd700 1 -- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc864159130 msgr2=0x7fc8640af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc862ffd700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc864159130 0x7fc8640af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.729 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc862ffd700 1 -- 172.21.15.84:0/119926670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85800b7f0 con 0x7fc864070770 2023-07-06T21:40:39.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc8637fe700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc864159130 0x7fc8640af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:39.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc862ffd700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 0x7fc8640c7760 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc85400d800 tx=0x7fc85400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:39.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8540125f0 con 0x7fc864070770 2023-07-06T21:40:39.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc854012750 con 0x7fc864070770 2023-07-06T21:40:39.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8640c69a0 con 0x7fc864070770 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854012b00 con 0x7fc864070770 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.721+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8640c6f60 con 0x7fc864070770 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.722+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8640c7480 con 0x7fc864070770 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.723+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc854012c60 con 0x7fc864070770 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.724+0000 7fc84b7fe700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc85006ace0 0x7fc85006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.724+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc8540898c0 con 0x7fc864070770 2023-07-06T21:40:39.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.724+0000 7fc8627fc700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc85006ace0 0x7fc85006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:39.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.725+0000 7fc8627fc700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc85006ace0 0x7fc85006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8640c5440 tx=0x7fc85801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:39.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.728+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc8640c7480 con 0x7fc864070770 2023-07-06T21:40:39.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.927+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc86415c770 con 0x7fc864070770 2023-07-06T21:40:39.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.928+0000 7fc84b7fe700 1 -- 172.21.15.84:0/119926670 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18374 (secure 0 0 0) 0x7fc854054890 con 0x7fc864070770 2023-07-06T21:40:39.932 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:39.932 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":25,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:07.972183+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:39:24.496506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-07-06T21:39:25.404949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.b","mappings":[{"from":1,"to":5}]},{"pgid":"4.28","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:39.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc85006ace0 msgr2=0x7fc85006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc85006ace0 0x7fc85006d190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8640c5440 tx=0x7fc85801c040 comp rx=0 tx=0).stop 2023-07-06T21:40:39.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 msgr2=0x7fc8640c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:39.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 0x7fc8640c7760 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc85400d800 tx=0x7fc85400db10 comp rx=0 tx=0).stop 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 shutdown_connections 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc85006ace0 0x7fc85006d190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc864159130 0x7fc8640af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc864070770 0x7fc8640c7760 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 --2- 172.21.15.84:0/119926670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8640c5000 0x7fc86415bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 >> 172.21.15.84:0/119926670 conn(0x7fc8640cf3d0 msgr2=0x7fc8640cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:39.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 shutdown_connections 2023-07-06T21:40:39.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:39.931+0000 7fc8693ff700 1 -- 172.21.15.84:0/119926670 wait complete. 2023-07-06T21:40:39.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T21:40:40.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.145+0000 7fafc73ff700 1 Processor -- start 2023-07-06T21:40:40.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.145+0000 7fafc73ff700 1 -- start start 2023-07-06T21:40:40.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc73ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00b83c0 0x7fafc00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:40.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc73ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc00d0a80 con 0x7fafc00d3710 2023-07-06T21:40:40.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc73ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc00b8790 con 0x7fafc00cfc60 2023-07-06T21:40:40.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc73ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00b4e70 con 0x7fafc00b83c0 2023-07-06T21:40:40.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc499a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fafc00d3710 0x7fafc00cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46528/0 (socket says 172.21.15.84:46528) 2023-07-06T21:40:40.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc499a700 1 -- 172.21.15.84:0/2563119140 learned_addr learned my addr 172.21.15.84:0/2563119140 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:40.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.146+0000 7fafc519b700 1 --2- 172.21.15.84:0/2563119140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00b83c0 0x7fafc00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:40.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880873812 0 0) 0x7fafc00d0a80 con 0x7fafc00d3710 2023-07-06T21:40:40.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafac0035b0 con 0x7fafc00d3710 2023-07-06T21:40:40.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451937909 0 0) 0x7fafc00b8790 con 0x7fafc00cfc60 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafc00d0a80 con 0x7fafc00cfc60 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafc519b700 1 -- 172.21.15.84:0/2563119140 >> v1:172.21.15.107:6789/0 conn(0x7fafc00cfc60 legacy=0x7fafc00d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1246983530 0 0) 0x7fafac0035b0 con 0x7fafc00d3710 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafc519b700 1 -- 172.21.15.84:0/2563119140 >> v1:172.21.15.84:6789/0 conn(0x7fafc00d3710 legacy=0x7fafc00cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafc519b700 1 -- 172.21.15.84:0/2563119140 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00b5150 con 0x7fafc00b83c0 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafc519b700 1 --2- 172.21.15.84:0/2563119140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00b83c0 0x7fafc00cd200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009870 tx=0x7fafbc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6591f14d0aabd7b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0001670 con 0x7fafc00b83c0 2023-07-06T21:40:40.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.147+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc004030 con 0x7fafc00b83c0 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafb7fff700 1 -- 172.21.15.84:0/2563119140 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0043c0 con 0x7fafc00b83c0 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 -- 172.21.15.84:0/2563119140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00b83c0 msgr2=0x7fafc00cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2563119140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00b83c0 0x7fafc00cd200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009870 tx=0x7fafbc009b80 comp rx=0 tx=0).stop 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 -- 172.21.15.84:0/2563119140 shutdown_connections 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2563119140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00b83c0 0x7fafc00cd200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 -- 172.21.15.84:0/2563119140 >> 172.21.15.84:0/2563119140 conn(0x7fafc006b350 msgr2=0x7fafc006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 -- 172.21.15.84:0/2563119140 shutdown_connections 2023-07-06T21:40:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.148+0000 7fafc73ff700 1 -- 172.21.15.84:0/2563119140 wait complete. 2023-07-06T21:40:40.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.150+0000 7fafc73ff700 1 Processor -- start 2023-07-06T21:40:40.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.151+0000 7fafc73ff700 1 -- start start 2023-07-06T21:40:40.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc73ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc00b83c0 0x7fafc0153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:40.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc73ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 0x7fafc015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:40.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc73ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00d3710 0x7fafc00d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:40.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc73ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00ca110 con 0x7fafc00cfc60 2023-07-06T21:40:40.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc73ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00ca3f0 con 0x7fafc00b83c0 2023-07-06T21:40:40.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc73ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00ca6d0 con 0x7fafc00d3710 2023-07-06T21:40:40.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc499a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 0x7fafc015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:40.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc499a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 0x7fafc015d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52498/0 (socket says 172.21.15.84:52498) 2023-07-06T21:40:40.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc499a700 1 -- 172.21.15.84:0/2727039338 learned_addr learned my addr 172.21.15.84:0/2727039338 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:40.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.152+0000 7fafc519b700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc00b83c0 0x7fafc0153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:40.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc599c700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00d3710 0x7fafc00d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:40.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc499a700 1 -- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00d3710 msgr2=0x7fafc00d29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc499a700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00d3710 0x7fafc00d29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc499a700 1 -- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc00b83c0 msgr2=0x7fafc0153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc499a700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc00b83c0 0x7fafc0153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc499a700 1 -- 172.21.15.84:0/2727039338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0094a0 con 0x7fafc00cfc60 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc519b700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc00b83c0 0x7fafc0153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc499a700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 0x7fafc015d270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c750 tx=0x7fafb000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:40.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0007560 con 0x7fafc00cfc60 2023-07-06T21:40:40.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb00076c0 con 0x7fafc00cfc60 2023-07-06T21:40:40.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc00ca9b0 con 0x7fafc00cfc60 2023-07-06T21:40:40.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0004a90 con 0x7fafc00cfc60 2023-07-06T21:40:40.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.153+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc0154400 con 0x7fafc00cfc60 2023-07-06T21:40:40.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.154+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc00caed0 con 0x7fafc00cfc60 2023-07-06T21:40:40.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.155+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fafb0004bf0 con 0x7fafc00cfc60 2023-07-06T21:40:40.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.156+0000 7fafb5ffb700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fafa406ace0 0x7fafa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:40.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.156+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fafb0088ab0 con 0x7fafc00cfc60 2023-07-06T21:40:40.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.156+0000 7fafc519b700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fafa406ace0 0x7fafa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:40.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.157+0000 7fafc519b700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fafa406ace0 0x7fafa406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0090b0 tx=0x7fafbc013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:40.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.160+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fafc00caed0 con 0x7fafc00cfc60 2023-07-06T21:40:40.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:40.399+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fafc00caed0 con 0x7fafc00cfc60 2023-07-06T21:40:41.104 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-07-06T21:40:41.101+0000 7feff05f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T21:40:41.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.108+0000 7fafb5ffb700 1 -- 172.21.15.84:0/2727039338 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fafc00caed0 con 0x7fafc00cfc60 2023-07-06T21:40:41.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.110+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fafa406ace0 msgr2=0x7fafa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:41.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.110+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fafa406ace0 0x7fafa406d190 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0090b0 tx=0x7fafbc013040 comp rx=0 tx=0).stop 2023-07-06T21:40:41.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.110+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 msgr2=0x7fafc015d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:41.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.110+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 0x7fafc015d270 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fafb000c750 tx=0x7fafb000cb10 comp rx=0 tx=0).stop 2023-07-06T21:40:41.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 shutdown_connections 2023-07-06T21:40:41.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fafa406ace0 0x7fafa406d190 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafc00d3710 0x7fafc00d29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafc00b83c0 0x7fafc0153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 --2- 172.21.15.84:0/2727039338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafc00cfc60 0x7fafc015d270 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 >> 172.21.15.84:0/2727039338 conn(0x7fafc006b350 msgr2=0x7fafc00b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 shutdown_connections 2023-07-06T21:40:41.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.111+0000 7fafc73ff700 1 -- 172.21.15.84:0/2727039338 wait complete. 2023-07-06T21:40:41.119 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T21:40:41.122 INFO:tasks.ceph.mds.d.smithi107.stderr: -3942> 2023-07-06T21:39:31.705+0000 7f4edf9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T21:40:41.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T21:40:41.176 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-06T21:40:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.326+0000 7f0fda9ff700 1 Processor -- start 2023-07-06T21:40:41.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.326+0000 7f0fda9ff700 1 -- start start 2023-07-06T21:40:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fda9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fd40b1a80 0x7f0fd40d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:41.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fda9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40d1830 con 0x7f0fd40b81c0 2023-07-06T21:40:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fda9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40cf330 con 0x7f0fd40b1a80 2023-07-06T21:40:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40d1270 con 0x7f0fd40cfd10 2023-07-06T21:40:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fd8f9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fd40b1a80 0x7f0fd40d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fd8f9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fd40b1a80 0x7f0fd40d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52508/0 (socket says 172.21.15.84:52508) 2023-07-06T21:40:41.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.327+0000 7f0fd8f9c700 1 -- 172.21.15.84:0/1198999535 learned_addr learned my addr 172.21.15.84:0/1198999535 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:41.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd37fe700 1 --2- 172.21.15.84:0/1198999535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd8f9c700 1 -- 172.21.15.84:0/1198999535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 msgr2=0x7f0fd40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd8f9c700 1 --2- 172.21.15.84:0/1198999535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd8f9c700 1 -- 172.21.15.84:0/1198999535 >> v1:172.21.15.107:6789/0 conn(0x7f0fd40b81c0 legacy=0x7f0fd40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd8f9c700 1 -- 172.21.15.84:0/1198999535 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd40d1550 con 0x7f0fd40b1a80 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd8f9c700 1 --2- 172.21.15.84:0/1198999535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b1a80 0x7f0fd40d28c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009e80 tx=0x7f0fcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab02ae4fcdfbce15 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd2ffd700 1 -- 172.21.15.84:0/1198999535 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc00e070 con 0x7f0fd40b1a80 2023-07-06T21:40:41.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd2ffd700 1 -- 172.21.15.84:0/1198999535 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fcc003a20 con 0x7f0fd40b1a80 2023-07-06T21:40:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.328+0000 7f0fd2ffd700 1 -- 172.21.15.84:0/1198999535 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc003dd0 con 0x7f0fd40b1a80 2023-07-06T21:40:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 -- 172.21.15.84:0/1198999535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b1a80 msgr2=0x7f0fd40d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/1198999535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b1a80 0x7f0fd40d28c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc009e80 tx=0x7f0fcc00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 -- 172.21.15.84:0/1198999535 shutdown_connections 2023-07-06T21:40:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/1198999535 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b1a80 0x7f0fd40d28c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/1198999535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 -- 172.21.15.84:0/1198999535 >> 172.21.15.84:0/1198999535 conn(0x7f0fd406b2b0 msgr2=0x7f0fd406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 -- 172.21.15.84:0/1198999535 shutdown_connections 2023-07-06T21:40:41.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.329+0000 7f0fda9ff700 1 -- 172.21.15.84:0/1198999535 wait complete. 2023-07-06T21:40:41.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.331+0000 7f0fda9ff700 1 Processor -- start 2023-07-06T21:40:41.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.331+0000 7f0fda9ff700 1 -- start start 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd40b1a80 0x7f0fd414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 0x7f0fd414a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fda9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd415d910 con 0x7f0fd40b81c0 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd414b0b0 con 0x7f0fd40b1a80 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fda9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd415ca20 con 0x7f0fd40cfd10 2023-07-06T21:40:41.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 0x7f0fd414a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 0x7f0fd414a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52512/0 (socket says 172.21.15.84:52512) 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 -- 172.21.15.84:0/2576475077 learned_addr learned my addr 172.21.15.84:0/2576475077 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd3fff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd40b1a80 0x7f0fd414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd8f9c700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 -- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 msgr2=0x7f0fd40caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 -- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd40b1a80 msgr2=0x7f0fd414ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:41.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd40b1a80 0x7f0fd414ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd37fe700 1 -- 172.21.15.84:0/2576475077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fcc009b30 con 0x7f0fd40b81c0 2023-07-06T21:40:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.333+0000 7f0fd3fff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd40b1a80 0x7f0fd414ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:41.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.334+0000 7f0fd37fe700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 0x7f0fd414a000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800d800 tx=0x7f0fc800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:41.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.334+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc80125f0 con 0x7f0fd40b81c0 2023-07-06T21:40:41.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.334+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8012750 con 0x7f0fd40b81c0 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.334+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd415cd00 con 0x7f0fd40b81c0 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.334+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8012ae0 con 0x7f0fd40b81c0 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.334+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd415d2c0 con 0x7f0fd40b81c0 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.335+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd404e1d0 con 0x7f0fd40b81c0 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.336+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0fc8012c40 con 0x7f0fd40b81c0 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.337+0000 7f0fd0ff9700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0fc006ace0 0x7f0fc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.338+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f0fc8089790 con 0x7f0fd40b81c0 2023-07-06T21:40:41.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.338+0000 7f0fd3fff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0fc006ace0 0x7f0fc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:41.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.338+0000 7f0fd3fff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0fc006ace0 0x7f0fc006d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4003fc0 tx=0x7f0fc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:41.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.342+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fc8054680 con 0x7f0fd40b81c0 2023-07-06T21:40:41.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:41.580+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0fd40b4f30 con 0x7f0fd40b81c0 2023-07-06T21:40:42.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.110+0000 7f0fd0ff9700 1 -- 172.21.15.84:0/2576475077 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0fc8057ca0 con 0x7f0fd40b81c0 2023-07-06T21:40:42.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0fc006ace0 msgr2=0x7f0fc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0fc006ace0 0x7f0fc006d190 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4003fc0 tx=0x7f0fc4009040 comp rx=0 tx=0).stop 2023-07-06T21:40:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 msgr2=0x7f0fd414a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 0x7f0fd414a000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0fc800d800 tx=0x7f0fc800db10 comp rx=0 tx=0).stop 2023-07-06T21:40:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 shutdown_connections 2023-07-06T21:40:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0fc006ace0 0x7f0fc006d190 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fd40cfd10 0x7f0fd40caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fd40b1a80 0x7f0fd414ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 --2- 172.21.15.84:0/2576475077 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fd40b81c0 0x7f0fd414a000 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 >> 172.21.15.84:0/2576475077 conn(0x7f0fd406b2b0 msgr2=0x7f0fd40732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 shutdown_connections 2023-07-06T21:40:42.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.112+0000 7f0fda9ff700 1 -- 172.21.15.84:0/2576475077 wait complete. 2023-07-06T21:40:42.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T21:40:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.330+0000 7fa31cbff700 1 Processor -- start 2023-07-06T21:40:42.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.330+0000 7fa31cbff700 1 -- start start 2023-07-06T21:40:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31cbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3180d0ea0 0x7fa3180d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31cbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa31815fee0 0x7fa3181602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:42.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31cbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3180ceef0 con 0x7fa318160ac0 2023-07-06T21:40:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31cbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3180cf450 con 0x7fa3180d0ea0 2023-07-06T21:40:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31cbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa318160820 con 0x7fa31815fee0 2023-07-06T21:40:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3180d0ea0 0x7fa3180d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3180d0ea0 0x7fa3180d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52514/0 (socket says 172.21.15.84:52514) 2023-07-06T21:40:42.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.331+0000 7fa31659c700 1 -- 172.21.15.84:0/3568166142 learned_addr learned my addr 172.21.15.84:0/3568166142 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa316d9d700 1 --2- 172.21.15.84:0/3568166142 >> v2:172.21.15.107:3300/0 conn(0x7fa31815fee0 0x7fa3181602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa31659c700 1 -- 172.21.15.84:0/3568166142 >> v2:172.21.15.107:3300/0 conn(0x7fa31815fee0 msgr2=0x7fa3181602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa31659c700 1 --2- 172.21.15.84:0/3568166142 >> v2:172.21.15.107:3300/0 conn(0x7fa31815fee0 0x7fa3181602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa31659c700 1 -- 172.21.15.84:0/3568166142 >> v1:172.21.15.107:6789/0 conn(0x7fa318160ac0 legacy=0x7fa3180cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa31659c700 1 -- 172.21.15.84:0/3568166142 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180cf120 con 0x7fa3180d0ea0 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa316d9d700 1 --2- 172.21.15.84:0/3568166142 >> v2:172.21.15.107:3300/0 conn(0x7fa31815fee0 0x7fa3181602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa31659c700 1 --2- 172.21.15.84:0/3568166142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3180d0ea0 0x7fa3180d1280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa3180c9d00 tx=0x7fa308009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e128ecb306fc9d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa315d9b700 1 -- 172.21.15.84:0/3568166142 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308005e30 con 0x7fa3180d0ea0 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa315d9b700 1 -- 172.21.15.84:0/3568166142 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa308006030 con 0x7fa3180d0ea0 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.332+0000 7fa315d9b700 1 -- 172.21.15.84:0/3568166142 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3080063e0 con 0x7fa3180d0ea0 2023-07-06T21:40:42.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 -- 172.21.15.84:0/3568166142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3180d0ea0 msgr2=0x7fa3180d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 --2- 172.21.15.84:0/3568166142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3180d0ea0 0x7fa3180d1280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa3180c9d00 tx=0x7fa308009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 -- 172.21.15.84:0/3568166142 shutdown_connections 2023-07-06T21:40:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 --2- 172.21.15.84:0/3568166142 >> v2:172.21.15.107:3300/0 conn(0x7fa31815fee0 0x7fa3181602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 --2- 172.21.15.84:0/3568166142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3180d0ea0 0x7fa3180d1280 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 -- 172.21.15.84:0/3568166142 >> 172.21.15.84:0/3568166142 conn(0x7fa3180715c0 msgr2=0x7fa318071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 -- 172.21.15.84:0/3568166142 shutdown_connections 2023-07-06T21:40:42.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.333+0000 7fa31cbff700 1 -- 172.21.15.84:0/3568166142 wait complete. 2023-07-06T21:40:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.335+0000 7fa31cbff700 1 Processor -- start 2023-07-06T21:40:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.335+0000 7fa31cbff700 1 -- start start 2023-07-06T21:40:42.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31cbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa31815fee0 0x7fa31813ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31cbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa318160ac0 0x7fa3181f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31cbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 0x7fa3181f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31cbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180c6790 con 0x7fa318142590 2023-07-06T21:40:42.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31cbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180c9650 con 0x7fa318160ac0 2023-07-06T21:40:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31cbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180c69e0 con 0x7fa31815fee0 2023-07-06T21:40:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 0x7fa3181f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:42.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 0x7fa3181f1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52528/0 (socket says 172.21.15.84:52528) 2023-07-06T21:40:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 -- 172.21.15.84:0/2275602820 learned_addr learned my addr 172.21.15.84:0/2275602820 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31659c700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa31815fee0 0x7fa31813ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa30ffff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa318160ac0 0x7fa3181f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:42.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 -- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa31815fee0 msgr2=0x7fa31813ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa31815fee0 0x7fa31813ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 -- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa318160ac0 msgr2=0x7fa3181f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa318160ac0 0x7fa3181f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:42.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa316d9d700 1 -- 172.21.15.84:0/2275602820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa308009500 con 0x7fa318142590 2023-07-06T21:40:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.336+0000 7fa31659c700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa31815fee0 0x7fa31813ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.337+0000 7fa316d9d700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 0x7fa3181f1580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa31000cef0 tx=0x7fa31000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.337+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000adb0 con 0x7fa318142590 2023-07-06T21:40:42.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.337+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310015440 con 0x7fa318142590 2023-07-06T21:40:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.337+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3181f1ac0 con 0x7fa318142590 2023-07-06T21:40:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.337+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3100157f0 con 0x7fa318142590 2023-07-06T21:40:42.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.337+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3180c6cc0 con 0x7fa318142590 2023-07-06T21:40:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.338+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31804e1d0 con 0x7fa318142590 2023-07-06T21:40:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.339+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa31000d430 con 0x7fa318142590 2023-07-06T21:40:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.340+0000 7fa30effd700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2f806ace0 0x7fa2f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.340+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fa310089a90 con 0x7fa318142590 2023-07-06T21:40:42.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.340+0000 7fa31659c700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2f806ace0 0x7fa2f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:42.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.341+0000 7fa31659c700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2f806ace0 0x7fa2f806d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3180c9d00 tx=0x7fa308008070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:42.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.344+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa310054aa0 con 0x7fa318142590 2023-07-06T21:40:42.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:42.541+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa3180d1930 con 0x7fa318142590 2023-07-06T21:40:43.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.115+0000 7fa30effd700 1 -- 172.21.15.84:0/2275602820 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa31001a0d0 con 0x7fa318142590 2023-07-06T21:40:43.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2f806ace0 msgr2=0x7fa2f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2f806ace0 0x7fa2f806d190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa3180c9d00 tx=0x7fa308008070 comp rx=0 tx=0).stop 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 msgr2=0x7fa3181f1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 0x7fa3181f1580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa31000cef0 tx=0x7fa31000cf20 comp rx=0 tx=0).stop 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 shutdown_connections 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa2f806ace0 0x7fa2f806d190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa318142590 0x7fa3181f1580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa31815fee0 0x7fa31813ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 --2- 172.21.15.84:0/2275602820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa318160ac0 0x7fa3181f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 >> 172.21.15.84:0/2275602820 conn(0x7fa3180715c0 msgr2=0x7fa3180cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.119+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 shutdown_connections 2023-07-06T21:40:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.120+0000 7fa31cbff700 1 -- 172.21.15.84:0/2275602820 wait complete. 2023-07-06T21:40:43.124 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-07-06T21:40:43.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T21:40:43.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.336+0000 7fc78dfff700 1 Processor -- start 2023-07-06T21:40:43.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.336+0000 7fc78dfff700 1 -- start start 2023-07-06T21:40:43.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.337+0000 7fc78dfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880b4590 0x7fc7880cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:43.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.337+0000 7fc78dfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7880cd7b0 con 0x7fc7880d2610 2023-07-06T21:40:43.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.337+0000 7fc78dfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7881609c0 con 0x7fc7880d0980 2023-07-06T21:40:43.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.337+0000 7fc78dfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc788160ca0 con 0x7fc7880b4590 2023-07-06T21:40:43.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.337+0000 7fc786ffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc7880d2610 0x7fc7880b4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42270/0 (socket says 172.21.15.84:42270) 2023-07-06T21:40:43.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.337+0000 7fc786ffd700 1 -- 172.21.15.84:0/1715160449 learned_addr learned my addr 172.21.15.84:0/1715160449 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7877fe700 1 --2- 172.21.15.84:0/1715160449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880b4590 0x7fc7880cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323589541 0 0) 0x7fc7880cd7b0 con 0x7fc7880d2610 2023-07-06T21:40:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7700035b0 con 0x7fc7880d2610 2023-07-06T21:40:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524053541 0 0) 0x7fc7881609c0 con 0x7fc7880d0980 2023-07-06T21:40:43.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7880cd7b0 con 0x7fc7880d0980 2023-07-06T21:40:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7877fe700 1 -- 172.21.15.84:0/1715160449 >> v1:172.21.15.107:6789/0 conn(0x7fc7880d0980 legacy=0x7fc7880d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7877fe700 1 -- 172.21.15.84:0/1715160449 >> v1:172.21.15.84:6789/0 conn(0x7fc7880d2610 legacy=0x7fc7880b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2816831755 0 0) 0x7fc7700035b0 con 0x7fc7880d2610 2023-07-06T21:40:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.338+0000 7fc7877fe700 1 -- 172.21.15.84:0/1715160449 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7880cd160 con 0x7fc7880b4590 2023-07-06T21:40:43.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc7877fe700 1 --2- 172.21.15.84:0/1715160449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880b4590 0x7fc7880cecc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc778009870 tx=0x7fc778009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9d2495f926bbfb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:43.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77c001670 con 0x7fc7880b4590 2023-07-06T21:40:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc778004030 con 0x7fc7880b4590 2023-07-06T21:40:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc7867fc700 1 -- 172.21.15.84:0/1715160449 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7780043c0 con 0x7fc7880b4590 2023-07-06T21:40:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc78dfff700 1 -- 172.21.15.84:0/1715160449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880b4590 msgr2=0x7fc7880cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc78dfff700 1 --2- 172.21.15.84:0/1715160449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880b4590 0x7fc7880cecc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc778009870 tx=0x7fc778009b80 comp rx=0 tx=0).stop 2023-07-06T21:40:43.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc78dfff700 1 -- 172.21.15.84:0/1715160449 shutdown_connections 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc78dfff700 1 --2- 172.21.15.84:0/1715160449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880b4590 0x7fc7880cecc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc78dfff700 1 -- 172.21.15.84:0/1715160449 >> 172.21.15.84:0/1715160449 conn(0x7fc7880cf590 msgr2=0x7fc7880cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.339+0000 7fc78dfff700 1 -- 172.21.15.84:0/1715160449 shutdown_connections 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.340+0000 7fc78dfff700 1 -- 172.21.15.84:0/1715160449 wait complete. 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.341+0000 7fc78dfff700 1 Processor -- start 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.341+0000 7fc78dfff700 1 -- start start 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc78dfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 0x7fc78815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:43.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc78dfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880d0980 0x7fc7880a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc78dfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7880d2610 0x7fc7880a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc78dfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880ac240 con 0x7fc7880b4590 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc78dfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78815aee0 con 0x7fc7880d2610 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc78dfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7880aa050 con 0x7fc7880d0980 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.342+0000 7fc7877fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 0x7fc78815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 0x7fc78815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54372/0 (socket says 172.21.15.84:54372) 2023-07-06T21:40:43.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 -- 172.21.15.84:0/748919951 learned_addr learned my addr 172.21.15.84:0/748919951 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc787fff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7880d2610 0x7fc7880a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc786ffd700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880d0980 0x7fc7880a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 -- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880d0980 msgr2=0x7fc7880a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880d0980 0x7fc7880a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 -- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7880d2610 msgr2=0x7fc7880a9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7880d2610 0x7fc7880a9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 -- 172.21.15.84:0/748919951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7780094a0 con 0x7fc7880b4590 2023-07-06T21:40:43.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc786ffd700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880d0980 0x7fc7880a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.343+0000 7fc7877fe700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 0x7fc78815a3b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc778004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.344+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778003d80 con 0x7fc7880b4590 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.344+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77801f070 con 0x7fc7880b4590 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.344+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7880aa330 con 0x7fc7880b4590 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.344+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778011870 con 0x7fc7880b4590 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.344+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7880aa8f0 con 0x7fc7880b4590 2023-07-06T21:40:43.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.345+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78815c5b0 con 0x7fc7880b4590 2023-07-06T21:40:43.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.346+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc778021030 con 0x7fc7880b4590 2023-07-06T21:40:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.347+0000 7fc76ffff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc77406ace0 0x7fc77406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.347+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5768+0+0 (secure 0 0 0) 0x7fc77808a5c0 con 0x7fc7880b4590 2023-07-06T21:40:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.347+0000 7fc786ffd700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc77406ace0 0x7fc77406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.348+0000 7fc786ffd700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc77406ace0 0x7fc77406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc77c001eb0 tx=0x7fc77c001ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:43.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.351+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc778055940 con 0x7fc7880b4590 2023-07-06T21:40:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:43.558+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc7880b3c60 con 0x7fc7880b4590 2023-07-06T21:40:44.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.117+0000 7fc76ffff700 1 -- 172.21.15.84:0/748919951 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc778058f60 con 0x7fc7880b4590 2023-07-06T21:40:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc77406ace0 msgr2=0x7fc77406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc77406ace0 0x7fc77406d190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc77c001eb0 tx=0x7fc77c001ca0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 msgr2=0x7fc78815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 0x7fc78815a3b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc778004750 comp rx=0 tx=0).stop 2023-07-06T21:40:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 shutdown_connections 2023-07-06T21:40:44.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc77406ace0 0x7fc77406d190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7880d2610 0x7fc7880a9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc7880b4590 0x7fc78815a3b0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc778000c00 tx=0x7fc778004750 comp rx=0 tx=0).stop 2023-07-06T21:40:44.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 --2- 172.21.15.84:0/748919951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7880d0980 0x7fc7880a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.121+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 >> 172.21.15.84:0/748919951 conn(0x7fc7880cf590 msgr2=0x7fc7880b7950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:44.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.122+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 shutdown_connections 2023-07-06T21:40:44.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.122+0000 7fc78dfff700 1 -- 172.21.15.84:0/748919951 wait complete. 2023-07-06T21:40:44.127 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-07-06T21:40:44.146 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T21:40:44.146 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T21:40:44.147 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T21:40:44.147 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:44.148 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T21:40:44.148 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T21:40:44.148 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T21:40:44.149 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:44.149 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T21:40:44.150 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T21:40:44.150 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T21:40:44.151 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:44.151 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T21:40:44.152 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T21:40:44.152 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T21:40:44.152 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:40:44.163 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T21:40:44.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T21:40:44.165 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T21:40:44.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T21:40:44.167 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T21:40:44.168 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T21:40:44.169 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T21:40:44.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T21:40:44.209 INFO:tasks.ceph.mds.a:Started 2023-07-06T21:40:44.209 DEBUG:teuthology.parallel:result is None 2023-07-06T21:40:44.212 INFO:tasks.ceph.mds.c:Started 2023-07-06T21:40:44.212 DEBUG:teuthology.parallel:result is None 2023-07-06T21:40:44.213 INFO:tasks.ceph.mds.b:Started 2023-07-06T21:40:44.214 DEBUG:teuthology.parallel:result is None 2023-07-06T21:40:44.217 INFO:tasks.ceph.mds.d:Started 2023-07-06T21:40:44.217 DEBUG:teuthology.parallel:result is None 2023-07-06T21:40:44.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T21:40:44.354 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-07-06T21:40:44.355 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-07-06T21:40:44.364 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2023-07-06T21:40:44.364 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-06T21:40:44.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.432+0000 7fbf6b7ff700 1 Processor -- start 2023-07-06T21:40:44.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.432+0000 7fbf6b7ff700 1 -- start start 2023-07-06T21:40:44.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.433+0000 7fbf6b7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf640b8530 0x7fbf640cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.433+0000 7fbf6b7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf6415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.433+0000 7fbf6b7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf640d0990 con 0x7fbf640d3580 2023-07-06T21:40:44.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.433+0000 7fbf6b7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640d3af0 con 0x7fbf640b8530 2023-07-06T21:40:44.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.433+0000 7fbf6b7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cd9d0 con 0x7fbf640d1580 2023-07-06T21:40:44.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf69d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf6415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:44.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf6959b700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf640b8530 0x7fbf640cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:44.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf68d9a700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fbf640d3580 0x7fbf640d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:47552/0 (socket says 172.21.15.84:47552) 2023-07-06T21:40:44.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf69d9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf6415c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:47228/0 (socket says 172.21.15.84:47228) 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf68d9a700 1 -- 172.21.15.84:0/2925211314 learned_addr learned my addr 172.21.15.84:0/2925211314 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf6959b700 1 -- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 msgr2=0x7fbf6415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf6959b700 1 --2- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf6415c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf6959b700 1 -- 172.21.15.84:0/2925211314 >> v1:172.21.15.107:6789/0 conn(0x7fbf640d3580 legacy=0x7fbf640d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf6959b700 1 -- 172.21.15.84:0/2925211314 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640cd410 con 0x7fbf640b8530 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf69d9c700 1 --2- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf6415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:44.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.434+0000 7fbf5bfff700 1 -- 172.21.15.84:0/2925211314 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3046707864 0 0) 0x7fbf640d0990 con 0x7fbf640d3580 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6959b700 1 --2- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640b8530 0x7fbf640cee70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009820 tx=0x7fbf60009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c439258876bd0cb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf5bfff700 1 -- 172.21.15.84:0/2925211314 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60005e30 con 0x7fbf640b8530 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf5bfff700 1 -- 172.21.15.84:0/2925211314 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60006030 con 0x7fbf640b8530 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf5bfff700 1 -- 172.21.15.84:0/2925211314 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600063c0 con 0x7fbf640b8530 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640b8530 msgr2=0x7fbf640cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640b8530 0x7fbf640cee70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009820 tx=0x7fbf60009b30 comp rx=0 tx=0).stop 2023-07-06T21:40:44.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/2925211314 shutdown_connections 2023-07-06T21:40:44.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf6415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/2925211314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640b8530 0x7fbf640cee70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.435+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/2925211314 >> 172.21.15.84:0/2925211314 conn(0x7fbf640715c0 msgr2=0x7fbf64071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:44.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.436+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/2925211314 shutdown_connections 2023-07-06T21:40:44.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.436+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/2925211314 wait complete. 2023-07-06T21:40:44.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.437+0000 7fbf6b7ff700 1 Processor -- start 2023-07-06T21:40:44.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.437+0000 7fbf6b7ff700 1 -- start start 2023-07-06T21:40:44.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6b7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 0x7fbf64154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:44.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6b7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf64146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6b7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640d3580 0x7fbf641472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6b7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640c77f0 con 0x7fbf640b8530 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6b7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf641477e0 con 0x7fbf640d3580 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6b7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640b31f0 con 0x7fbf640d1580 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.438+0000 7fbf6959b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 0x7fbf64154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 0x7fbf64154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54424/0 (socket says 172.21.15.84:54424) 2023-07-06T21:40:44.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 -- 172.21.15.84:0/3919269333 learned_addr learned my addr 172.21.15.84:0/3919269333 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf68d9a700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf64146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf69d9c700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640d3580 0x7fbf641472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 -- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 msgr2=0x7fbf64146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf64146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 -- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640d3580 msgr2=0x7fbf641472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640d3580 0x7fbf641472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:44.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 -- 172.21.15.84:0/3919269333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c009a70 con 0x7fbf640b8530 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf69d9c700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640d3580 0x7fbf641472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6959b700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 0x7fbf64154c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c00 tx=0x7fbf60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf600061a0 con 0x7fbf640b8530 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf60020070 con 0x7fbf640b8530 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf60006540 con 0x7fbf640b8530 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.439+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf60009430 con 0x7fbf640b8530 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.440+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf640b37b0 con 0x7fbf640b8530 2023-07-06T21:40:44.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.441+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf640b3dc0 con 0x7fbf640b8530 2023-07-06T21:40:44.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.442+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbf60022030 con 0x7fbf640b8530 2023-07-06T21:40:44.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.442+0000 7fbf59ffb700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbf4806ace0 0x7fbf4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:44.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.443+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbf6008a5a0 con 0x7fbf640b8530 2023-07-06T21:40:44.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.443+0000 7fbf68d9a700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbf4806ace0 0x7fbf4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:44.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.443+0000 7fbf68d9a700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbf4806ace0 0x7fbf4806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf641f56c0 tx=0x7fbf54007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:44.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.447+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbf600559e0 con 0x7fbf640b8530 2023-07-06T21:40:44.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:44.662+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbf640cd6f0 con 0x7fbf640b8530 2023-07-06T21:40:45.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.403+0000 7fbf59ffb700 1 -- 172.21.15.84:0/3919269333 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf60059000 con 0x7fbf640b8530 2023-07-06T21:40:45.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbf4806ace0 msgr2=0x7fbf4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbf4806ace0 0x7fbf4806d190 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf641f56c0 tx=0x7fbf54007e70 comp rx=0 tx=0).stop 2023-07-06T21:40:45.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 msgr2=0x7fbf64154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 0x7fbf64154c50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbf60000c00 tx=0x7fbf60004750 comp rx=0 tx=0).stop 2023-07-06T21:40:45.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 shutdown_connections 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbf4806ace0 0x7fbf4806d190 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf640d3580 0x7fbf641472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbf640b8530 0x7fbf64154c50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 --2- 172.21.15.84:0/3919269333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf640d1580 0x7fbf64146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 >> 172.21.15.84:0/3919269333 conn(0x7fbf640715c0 msgr2=0x7fbf640d2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 shutdown_connections 2023-07-06T21:40:45.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.406+0000 7fbf6b7ff700 1 -- 172.21.15.84:0/3919269333 wait complete. 2023-07-06T21:40:45.411 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-07-06T21:40:45.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T21:40:45.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.620+0000 7fa067dff700 1 Processor -- start 2023-07-06T21:40:45.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.620+0000 7fa067dff700 1 -- start start 2023-07-06T21:40:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.621+0000 7fa067dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0600b4d90 0x7fa0600b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.621+0000 7fa067dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0600d1860 con 0x7fa06015c2c0 2023-07-06T21:40:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.621+0000 7fa067dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0600cf4c0 con 0x7fa06015c690 2023-07-06T21:40:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.621+0000 7fa067dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0600d1580 con 0x7fa0600b4d90 2023-07-06T21:40:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa065b9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa06015c2c0 0x7fa0600cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42290/0 (socket says 172.21.15.84:42290) 2023-07-06T21:40:45.626 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa065b9b700 1 -- 172.21.15.84:0/3254745506 learned_addr learned my addr 172.21.15.84:0/3254745506 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:45.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa06639c700 1 --2- 172.21.15.84:0/3254745506 >> v2:172.21.15.107:3300/0 conn(0x7fa0600b4d90 0x7fa0600b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.633 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511938059 0 0) 0x7fa0600d1860 con 0x7fa06015c2c0 2023-07-06T21:40:45.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa04c0035b0 con 0x7fa06015c2c0 2023-07-06T21:40:45.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1415901875 0 0) 0x7fa04c0035b0 con 0x7fa06015c2c0 2023-07-06T21:40:45.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 >> v2:172.21.15.107:3300/0 conn(0x7fa0600b4d90 msgr2=0x7fa0600b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:45.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 --2- 172.21.15.84:0/3254745506 >> v2:172.21.15.107:3300/0 conn(0x7fa0600b4d90 0x7fa0600b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 >> v1:172.21.15.107:6789/0 conn(0x7fa06015c690 legacy=0x7fa0600b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0600cd380 con 0x7fa06015c2c0 2023-07-06T21:40:45.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.622+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fa05c001670 con 0x7fa06015c2c0 2023-07-06T21:40:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 >> v1:172.21.15.84:6789/0 conn(0x7fa06015c2c0 legacy=0x7fa0600cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c003a00 0x7fa04c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa04c004850 0x7fa04c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa04c005200 0x7fa04c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600d1580 con 0x7fa04c005200 2023-07-06T21:40:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c00c110 con 0x7fa04c004850 2023-07-06T21:40:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c00c2d0 con 0x7fa04c003a00 2023-07-06T21:40:45.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa04c005200 0x7fa04c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa065b9b700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c003a00 0x7fa04c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06539a700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa04c004850 0x7fa04c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 -- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c003a00 msgr2=0x7fa04c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c003a00 0x7fa04c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 -- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa04c004850 msgr2=0x7fa04c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa04c004850 0x7fa04c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 -- 172.21.15.84:0/3254745506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0035b0 con 0x7fa04c005200 2023-07-06T21:40:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.623+0000 7fa06639c700 1 -- 172.21.15.84:0/3254745506 reap_dead start 2023-07-06T21:40:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa06639c700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa04c005200 0x7fa04c00ba10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa058009f90 tx=0x7fa04c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d78f7336b7835dd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:45.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058003a20 con 0x7fa04c005200 2023-07-06T21:40:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05800f040 con 0x7fa04c005200 2023-07-06T21:40:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa064b99700 1 -- 172.21.15.84:0/3254745506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa058003dc0 con 0x7fa04c005200 2023-07-06T21:40:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa067dff700 1 -- 172.21.15.84:0/3254745506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa04c005200 msgr2=0x7fa04c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa067dff700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa04c005200 0x7fa04c00ba10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa058009f90 tx=0x7fa04c005880 comp rx=0 tx=0).stop 2023-07-06T21:40:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa067dff700 1 -- 172.21.15.84:0/3254745506 shutdown_connections 2023-07-06T21:40:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa067dff700 1 --2- 172.21.15.84:0/3254745506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa04c005200 0x7fa04c00ba10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa067dff700 1 -- 172.21.15.84:0/3254745506 >> 172.21.15.84:0/3254745506 conn(0x7fa0600cf680 msgr2=0x7fa0600cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:45.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.624+0000 7fa067dff700 1 -- 172.21.15.84:0/3254745506 shutdown_connections 2023-07-06T21:40:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.625+0000 7fa067dff700 1 -- 172.21.15.84:0/3254745506 wait complete. 2023-07-06T21:40:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.626+0000 7fa067dff700 1 Processor -- start 2023-07-06T21:40:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.626+0000 7fa067dff700 1 -- start start 2023-07-06T21:40:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.627+0000 7fa067dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c005200 0x7fa060150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.627+0000 7fa067dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 0x7fa06015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.627+0000 7fa067dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060150d50 0x7fa06015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.627+0000 7fa067dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060150b90 con 0x7fa0600c96d0 2023-07-06T21:40:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.627+0000 7fa067dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600c70c0 con 0x7fa060150d50 2023-07-06T21:40:45.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.627+0000 7fa067dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0600b8780 con 0x7fa04c005200 2023-07-06T21:40:45.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 0x7fa06015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 0x7fa06015df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54450/0 (socket says 172.21.15.84:54450) 2023-07-06T21:40:45.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 -- 172.21.15.84:0/1545724980 learned_addr learned my addr 172.21.15.84:0/1545724980 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:45.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa065b9b700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c005200 0x7fa060150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06639c700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060150d50 0x7fa06015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 -- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c005200 msgr2=0x7fa060150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c005200 0x7fa060150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 -- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060150d50 msgr2=0x7fa06015e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060150d50 0x7fa06015e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 -- 172.21.15.84:0/1545724980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0600c7330 con 0x7fa0600c96d0 2023-07-06T21:40:45.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa06539a700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 0x7fa06015df70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa05000f820 tx=0x7fa05000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:45.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500052e0 con 0x7fa0600c96d0 2023-07-06T21:40:45.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.628+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa050005440 con 0x7fa0600c96d0 2023-07-06T21:40:45.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.629+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0601f23a0 con 0x7fa0600c96d0 2023-07-06T21:40:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.629+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0500038c0 con 0x7fa0600c96d0 2023-07-06T21:40:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.629+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0600c7610 con 0x7fa0600c96d0 2023-07-06T21:40:45.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.630+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0600c77a0 con 0x7fa0600c96d0 2023-07-06T21:40:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.631+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa050003a20 con 0x7fa0600c96d0 2023-07-06T21:40:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.632+0000 7fa0567fc700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa04406ace0 0x7fa04406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:45.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.632+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fa050088d60 con 0x7fa0600c96d0 2023-07-06T21:40:45.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.632+0000 7fa065b9b700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa04406ace0 0x7fa04406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:45.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.633+0000 7fa065b9b700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa04406ace0 0x7fa04406d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa05c007d10 tx=0x7fa05c000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:45.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.636+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa0600c77a0 con 0x7fa0600c96d0 2023-07-06T21:40:45.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.913+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa0600b5a50 con 0x7fa0600c96d0 2023-07-06T21:40:45.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.913+0000 7fa0567fc700 1 -- 172.21.15.84:0/1545724980 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fa0600b5a50 con 0x7fa0600c96d0 2023-07-06T21:40:45.917 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:45.917 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:40:45.917 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-07-06T21:40:45.917 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.917 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-07-06T21:40:45.917 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdk+/m1ERAAVuVl/rTq2tc0Dhv83fjupA==", 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.918 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.919 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-07-06T21:40:45.919 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdkRtxvKRAAIt8IaHnqNpNdKNcNdnoN7g==", 2023-07-06T21:40:45.919 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.919 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.919 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:40:45.920 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:40:45.920 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.920 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.920 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.920 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-07-06T21:40:45.921 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdkpgiGHRAA+dmQ4tCjJggxC0UEAx4E8A==", 2023-07-06T21:40:45.921 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.921 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.921 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:40:45.921 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:40:45.921 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdkdtB1NRAAZPgJEifPgIZTnZI5Sr0ZIw==", 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.922 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.923 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:40:45.923 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:40:45.923 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.923 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.924 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.924 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-07-06T21:40:45.924 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBRNKdko5ENMxAAkg9Bb14zud5gryZG5cNFmA==", 2023-07-06T21:40:45.924 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.925 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.925 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.925 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.925 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBUNKdkIAyGOhAAGm6K8mPZUi+KIOhlHHcf/A==", 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.926 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.927 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.927 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.927 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.927 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-07-06T21:40:45.928 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBYNKdkPYfnBBAAQiGXT+pTVZ/rM8G100/dGQ==", 2023-07-06T21:40:45.928 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.928 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.928 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.929 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.929 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.929 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.929 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.929 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-07-06T21:40:45.929 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBbNKdklfeTCxAAEMCI574RRSmAnzz2hU00Jg==", 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.930 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.931 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-07-06T21:40:45.931 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBhNKdkluR8IxAAd2WQtPkTN09bIdqHrbRBCg==", 2023-07-06T21:40:45.931 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.931 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.931 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.931 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.932 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.932 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.932 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.932 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-07-06T21:40:45.933 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBkNKdkH7wwKRAAyXE0BPwP8/koUUWCcCA1dQ==", 2023-07-06T21:40:45.933 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.933 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.933 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.934 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.934 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.934 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.935 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-07-06T21:40:45.935 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBnNKdk363RMBAArv8mMHQtqZRUf3AN2PqwTQ==", 2023-07-06T21:40:45.935 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.936 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.936 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.936 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.937 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.937 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.937 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.937 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-07-06T21:40:45.938 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBqNKdkTpaXOBAACmQ850h7EOrJ8sl0ODNYmA==", 2023-07-06T21:40:45.938 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.938 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:40:45.939 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:40:45.939 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.939 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.940 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.940 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.940 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-07-06T21:40:45.940 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdkqmQnBBAALedUX3/ZDQ9mAhOGEH6QrA==", 2023-07-06T21:40:45.941 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.941 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.941 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:40:45.942 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:40:45.942 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:40:45.942 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.942 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdkLt+uExAAzCoF/xhx1jkry5bkwObTEQ==", 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:40:45.943 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:40:45.944 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:40:45.944 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.944 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.944 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-07-06T21:40:45.945 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdk0bfQCxAA+4nQgIo/iEDM930/C0JV1w==", 2023-07-06T21:40:45.945 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.945 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.945 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:40:45.946 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:40:45.946 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:40:45.946 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.946 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.946 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-07-06T21:40:45.947 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdkodhbGxAA8n9L0VrpLjBlrDyEi4Sztg==", 2023-07-06T21:40:45.947 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.948 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:40:45.948 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:40:45.948 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:40:45.948 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:40:45.948 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.948 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBMNKdklNf3ExAAg1unuhVMRgLZ3q8jZCwR0g==", 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:40:45.949 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:40:45.950 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.950 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.950 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.950 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-mds", 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQByNKdkub3YNhAASrdWZCiZujBa6jD7EdaVVA==", 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-mds" 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.951 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-mgr", 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQByNKdkIcfYNhAAQ0VgtK6m4jGAmjC2ssqhQg==", 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-mgr" 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.952 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-osd", 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQByNKdkAdDYNhAAa+ObM5p+JLitIjvlwwRrdw==", 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-osd" 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.953 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rbd", 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQByNKdkldjYNhAAhnitSIFmICXmwVAqWTMB3w==", 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rbd" 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.954 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-07-06T21:40:45.955 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQByNKdkAuHYNhAA/LsGZe251yjSw3nXBZA4wQ==", 2023-07-06T21:40:45.955 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.955 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-07-06T21:40:45.955 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.955 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.956 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.bootstrap-rgw", 2023-07-06T21:40:45.956 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQByNKdk0evYNhAAvDfwOuXJ903iqs/HZTt/5Q==", 2023-07-06T21:40:45.956 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.956 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile bootstrap-rgw" 2023-07-06T21:40:45.956 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.957 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.957 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-07-06T21:40:45.957 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdk8JijCRAAGnCGHPatWKPv0GPKCjXNtg==", 2023-07-06T21:40:45.957 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.957 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-07-06T21:40:45.958 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-07-06T21:40:45.958 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.958 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:45.958 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:45.958 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBMNKdkfcUzOxAA9BUPVthEZwBTmBTqdmDopg==", 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.959 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:45.960 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-07-06T21:40:45.960 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:40:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa04406ace0 msgr2=0x7fa04406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa04406ace0 0x7fa04406d190 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa05c007d10 tx=0x7fa05c000f80 comp rx=0 tx=0).stop 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 msgr2=0x7fa06015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 0x7fa06015df70 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa05000f820 tx=0x7fa05000fb30 comp rx=0 tx=0).stop 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 shutdown_connections 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa04406ace0 0x7fa04406d190 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060150d50 0x7fa06015e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa04c005200 0x7fa060150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 --2- 172.21.15.84:0/1545724980 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0600c96d0 0x7fa06015df70 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.915+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 >> 172.21.15.84:0/1545724980 conn(0x7fa0600cf680 msgr2=0x7fa0600cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.916+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 shutdown_connections 2023-07-06T21:40:45.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:45.916+0000 7fa067dff700 1 -- 172.21.15.84:0/1545724980 wait complete. 2023-07-06T21:40:45.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-07-06T21:40:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.156+0000 7fc8eefff700 1 Processor -- start 2023-07-06T21:40:46.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.156+0000 7fc8eefff700 1 -- start start 2023-07-06T21:40:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8eefff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80b83c0 0x7fc8e80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8eefff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8e80d3710 0x7fc8e80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8eefff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e80cfee0 con 0x7fc8e80cfa90 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8eefff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e80d0a80 con 0x7fc8e80d3710 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8eefff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80b8790 con 0x7fc8e80b83c0 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8e7fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8e80d3710 0x7fc8e80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8e7fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc8e80d3710 0x7fc8e80cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54460/0 (socket says 172.21.15.84:54460) 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8e7fff700 1 -- 172.21.15.84:0/1762639268 learned_addr learned my addr 172.21.15.84:0/1762639268 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:46.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.157+0000 7fc8ecd9b700 1 --2- 172.21.15.84:0/1762639268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80b83c0 0x7fc8e80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e7fff700 1 -- 172.21.15.84:0/1762639268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80b83c0 msgr2=0x7fc8e80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e7fff700 1 --2- 172.21.15.84:0/1762639268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80b83c0 0x7fc8e80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e7fff700 1 -- 172.21.15.84:0/1762639268 >> v1:172.21.15.107:6789/0 conn(0x7fc8e80cfa90 legacy=0x7fc8e815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e7fff700 1 -- 172.21.15.84:0/1762639268 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80b4e70 con 0x7fc8e80d3710 2023-07-06T21:40:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e7fff700 1 --2- 172.21.15.84:0/1762639268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80d3710 0x7fc8e80cf550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40098f0 tx=0x7fc8d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee2ddfaa52d13e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e77fe700 1 -- 172.21.15.84:0/1762639268 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011830 con 0x7fc8e80d3710 2023-07-06T21:40:46.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e77fe700 1 -- 172.21.15.84:0/1762639268 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4011990 con 0x7fc8e80d3710 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.158+0000 7fc8e77fe700 1 -- 172.21.15.84:0/1762639268 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4011d20 con 0x7fc8e80d3710 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 -- 172.21.15.84:0/1762639268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80d3710 msgr2=0x7fc8e80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 --2- 172.21.15.84:0/1762639268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80d3710 0x7fc8e80cf550 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40098f0 tx=0x7fc8d4009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 -- 172.21.15.84:0/1762639268 shutdown_connections 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 --2- 172.21.15.84:0/1762639268 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80b83c0 0x7fc8e80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 --2- 172.21.15.84:0/1762639268 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80d3710 0x7fc8e80cf550 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 -- 172.21.15.84:0/1762639268 >> 172.21.15.84:0/1762639268 conn(0x7fc8e806b350 msgr2=0x7fc8e806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 -- 172.21.15.84:0/1762639268 shutdown_connections 2023-07-06T21:40:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.159+0000 7fc8eefff700 1 -- 172.21.15.84:0/1762639268 wait complete. 2023-07-06T21:40:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.161+0000 7fc8eefff700 1 Processor -- start 2023-07-06T21:40:46.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.161+0000 7fc8eefff700 1 -- start start 2023-07-06T21:40:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8eefff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 0x7fc8e8151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8eefff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80cfa90 0x7fc8e81f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8eefff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e80d3710 0x7fc8e8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8eefff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e815d6c0 con 0x7fc8e80b83c0 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8eefff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8151590 con 0x7fc8e80d3710 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8eefff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e815cab0 con 0x7fc8e80cfa90 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 0x7fc8e8151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 0x7fc8e8151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54476/0 (socket says 172.21.15.84:54476) 2023-07-06T21:40:46.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 -- 172.21.15.84:0/2897880846 learned_addr learned my addr 172.21.15.84:0/2897880846 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ed59c700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e80d3710 0x7fc8e8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8e7fff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80cfa90 0x7fc8e81f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 -- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80cfa90 msgr2=0x7fc8e81f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80cfa90 0x7fc8e81f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 -- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e80d3710 msgr2=0x7fc8e8159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e80d3710 0x7fc8e8159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.163+0000 7fc8ecd9b700 1 -- 172.21.15.84:0/2897880846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4009550 con 0x7fc8e80b83c0 2023-07-06T21:40:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.164+0000 7fc8ecd9b700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 0x7fc8e8151050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00e870 tx=0x7fc8dc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.164+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc009590 con 0x7fc8e80b83c0 2023-07-06T21:40:46.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.164+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc00ee70 con 0x7fc8e80b83c0 2023-07-06T21:40:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.164+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e815cd90 con 0x7fc8e80b83c0 2023-07-06T21:40:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.164+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc00ca50 con 0x7fc8e80b83c0 2023-07-06T21:40:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.164+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e815d350 con 0x7fc8e80b83c0 2023-07-06T21:40:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.165+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e80c63f0 con 0x7fc8e80b83c0 2023-07-06T21:40:46.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.166+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc8dc00cbb0 con 0x7fc8e80b83c0 2023-07-06T21:40:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.167+0000 7fc8e57fa700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc8cc06ad10 0x7fc8cc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.167+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc8dc089130 con 0x7fc8e80b83c0 2023-07-06T21:40:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.167+0000 7fc8e7fff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc8cc06ad10 0x7fc8cc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.168+0000 7fc8e7fff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc8cc06ad10 0x7fc8cc06d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80d3b20 tx=0x7fc8d4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:46.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.171+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc8dc054490 con 0x7fc8e80b83c0 2023-07-06T21:40:46.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.445+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fc8e80d0f00 con 0x7fc8e80b83c0 2023-07-06T21:40:46.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.449+0000 7fc8e57fa700 1 -- 172.21.15.84:0/2897880846 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc8dc00ce60 con 0x7fc8e80b83c0 2023-07-06T21:40:46.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.451+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc8cc06ad10 msgr2=0x7fc8cc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.451+0000 7fc8eefff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc8cc06ad10 0x7fc8cc06d1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80d3b20 tx=0x7fc8d4008030 comp rx=0 tx=0).stop 2023-07-06T21:40:46.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 msgr2=0x7fc8e8151050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 0x7fc8e8151050 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00e870 tx=0x7fc8dc00ec30 comp rx=0 tx=0).stop 2023-07-06T21:40:46.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 shutdown_connections 2023-07-06T21:40:46.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc8cc06ad10 0x7fc8cc06d1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e80d3710 0x7fc8e8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8e80b83c0 0x7fc8e8151050 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 --2- 172.21.15.84:0/2897880846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e80cfa90 0x7fc8e81f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 >> 172.21.15.84:0/2897880846 conn(0x7fc8e806b350 msgr2=0x7fc8e80d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:46.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 shutdown_connections 2023-07-06T21:40:46.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.452+0000 7fc8eefff700 1 -- 172.21.15.84:0/2897880846 wait complete. 2023-07-06T21:40:46.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-07-06T21:40:46.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.664+0000 7fdced3ff700 1 Processor -- start 2023-07-06T21:40:46.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.664+0000 7fdced3ff700 1 -- start start 2023-07-06T21:40:46.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.665+0000 7fdced3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdce80b83c0 0x7fdce80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.665+0000 7fdced3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80cfc60 0x7fdce80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.665+0000 7fdced3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce80d0a80 con 0x7fdce80d3710 2023-07-06T21:40:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.665+0000 7fdced3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce80b8790 con 0x7fdce80b83c0 2023-07-06T21:40:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.665+0000 7fdced3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80b4e70 con 0x7fdce80cfc60 2023-07-06T21:40:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce67fc700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdce80d3710 0x7fdce80cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42302/0 (socket says 172.21.15.84:42302) 2023-07-06T21:40:46.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce67fc700 1 -- 172.21.15.84:0/1872976369 learned_addr learned my addr 172.21.15.84:0/1872976369 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/1872976369 >> v2:172.21.15.107:3300/0 conn(0x7fdce80b83c0 0x7fdce80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce77fe700 1 --2- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80cfc60 0x7fdce80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce5ffb700 1 -- 172.21.15.84:0/1872976369 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701695602 0 0) 0x7fdce80d0a80 con 0x7fdce80d3710 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce5ffb700 1 -- 172.21.15.84:0/1872976369 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd00035b0 con 0x7fdce80d3710 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce6ffd700 1 -- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80cfc60 msgr2=0x7fdce80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80cfc60 0x7fdce80d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce6ffd700 1 -- 172.21.15.84:0/1872976369 >> v1:172.21.15.84:6789/0 conn(0x7fdce80d3710 legacy=0x7fdce80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce5ffb700 1 -- 172.21.15.84:0/1872976369 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 235259781 0 0) 0x7fdcd00035b0 con 0x7fdce80d3710 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.666+0000 7fdce6ffd700 1 -- 172.21.15.84:0/1872976369 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80b5150 con 0x7fdce80b83c0 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80b83c0 0x7fdce80cd200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009840 tx=0x7fdcd8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9545cf0fdf256f49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdce5ffb700 1 -- 172.21.15.84:0/1872976369 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc001670 con 0x7fdce80b83c0 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdce5ffb700 1 -- 172.21.15.84:0/1872976369 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd8004030 con 0x7fdce80b83c0 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdce5ffb700 1 -- 172.21.15.84:0/1872976369 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80043c0 con 0x7fdce80b83c0 2023-07-06T21:40:46.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdced3ff700 1 -- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80b83c0 msgr2=0x7fdce80cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdced3ff700 1 --2- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80b83c0 0x7fdce80cd200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009840 tx=0x7fdcd8009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdced3ff700 1 -- 172.21.15.84:0/1872976369 shutdown_connections 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdced3ff700 1 --2- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80cfc60 0x7fdce80d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.667+0000 7fdced3ff700 1 --2- 172.21.15.84:0/1872976369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80b83c0 0x7fdce80cd200 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.668+0000 7fdced3ff700 1 -- 172.21.15.84:0/1872976369 >> 172.21.15.84:0/1872976369 conn(0x7fdce806b350 msgr2=0x7fdce806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.668+0000 7fdced3ff700 1 -- 172.21.15.84:0/1872976369 shutdown_connections 2023-07-06T21:40:46.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.668+0000 7fdced3ff700 1 -- 172.21.15.84:0/1872976369 wait complete. 2023-07-06T21:40:46.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.669+0000 7fdced3ff700 1 Processor -- start 2023-07-06T21:40:46.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.669+0000 7fdced3ff700 1 -- start start 2023-07-06T21:40:46.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdced3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80b83c0 0x7fdce815f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdced3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 0x7fdce815bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdced3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80d3710 0x7fdce8153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdced3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8154400 con 0x7fdce80cfc60 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdced3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80cb630 con 0x7fdce80d3710 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdced3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80cad00 con 0x7fdce80b83c0 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.670+0000 7fdce67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 0x7fdce815bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 0x7fdce815bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54482/0 (socket says 172.21.15.84:54482) 2023-07-06T21:40:46.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 -- 172.21.15.84:0/2637746681 learned_addr learned my addr 172.21.15.84:0/2637746681 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce77fe700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80d3710 0x7fdce8153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80b83c0 0x7fdce815f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 -- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80b83c0 msgr2=0x7fdce815f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80b83c0 0x7fdce815f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 -- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80d3710 msgr2=0x7fdce8153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80d3710 0x7fdce8153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 -- 172.21.15.84:0/2637746681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd80094a0 con 0x7fdce80cfc60 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80b83c0 0x7fdce815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce77fe700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80d3710 0x7fdce8153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdce67fc700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 0x7fdce815bb40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00c780 tx=0x7fdcdc00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc004c80 con 0x7fdce80cfc60 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc004de0 con 0x7fdce80cfc60 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce80cafe0 con 0x7fdce80cfc60 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.671+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0072d0 con 0x7fdce80cfc60 2023-07-06T21:40:46.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.672+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce815f790 con 0x7fdce80cfc60 2023-07-06T21:40:46.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.673+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce80d1250 con 0x7fdce80cfc60 2023-07-06T21:40:46.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.673+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdcdc007430 con 0x7fdce80cfc60 2023-07-06T21:40:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.674+0000 7fdccf7fe700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdcd406ace0 0x7fdcd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.674+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fdcdc088840 con 0x7fdce80cfc60 2023-07-06T21:40:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.675+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdcd406ace0 0x7fdcd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:46.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.675+0000 7fdce6ffd700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdcd406ace0 0x7fdcd406d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009b90 tx=0x7fdcd8013040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:46.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.679+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdcdc053ba0 con 0x7fdce80cfc60 2023-07-06T21:40:46.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.955+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fdce804e240 con 0x7fdce80cfc60 2023-07-06T21:40:46.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.959+0000 7fdccf7fe700 1 -- 172.21.15.84:0/2637746681 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdcdc0571c0 con 0x7fdce80cfc60 2023-07-06T21:40:46.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdcd406ace0 msgr2=0x7fdcd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdcd406ace0 0x7fdcd406d190 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009b90 tx=0x7fdcd8013040 comp rx=0 tx=0).stop 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 msgr2=0x7fdce815bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 0x7fdce815bb40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00c780 tx=0x7fdcdc00cb40 comp rx=0 tx=0).stop 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 shutdown_connections 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdcd406ace0 0x7fdcd406d190 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce80d3710 0x7fdce8153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdce80b83c0 0x7fdce815f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 --2- 172.21.15.84:0/2637746681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdce80cfc60 0x7fdce815bb40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:46.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 >> 172.21.15.84:0/2637746681 conn(0x7fdce806b350 msgr2=0x7fdce815a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:46.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 shutdown_connections 2023-07-06T21:40:46.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:46.961+0000 7fdced3ff700 1 -- 172.21.15.84:0/2637746681 wait complete. 2023-07-06T21:40:46.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-07-06T21:40:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.174+0000 7f05719ff700 1 Processor -- start 2023-07-06T21:40:47.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.174+0000 7f05719ff700 1 -- start start 2023-07-06T21:40:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.175+0000 7f05719ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f056c0b83c0 0x7f056c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.175+0000 7f05719ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f056c0cfc60 0x7f056c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.175+0000 7f05719ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056c0d0a80 con 0x7f056c0d3710 2023-07-06T21:40:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.175+0000 7f05719ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c0b8790 con 0x7f056c0cfc60 2023-07-06T21:40:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.175+0000 7f05719ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f056c0b4e70 con 0x7f056c0b83c0 2023-07-06T21:40:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.175+0000 7f056b7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f056c0cfc60 0x7f056c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056a7fc700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f056c0d3710 0x7f056c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42318/0 (socket says 172.21.15.84:42318) 2023-07-06T21:40:47.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056a7fc700 1 -- 172.21.15.84:0/613741065 learned_addr learned my addr 172.21.15.84:0/613741065 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056affd700 1 --2- 172.21.15.84:0/613741065 >> v2:172.21.15.107:3300/0 conn(0x7f056c0b83c0 0x7f056c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056b7fe700 1 -- 172.21.15.84:0/613741065 >> v2:172.21.15.107:3300/0 conn(0x7f056c0b83c0 msgr2=0x7f056c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056b7fe700 1 --2- 172.21.15.84:0/613741065 >> v2:172.21.15.107:3300/0 conn(0x7f056c0b83c0 0x7f056c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056b7fe700 1 -- 172.21.15.84:0/613741065 >> v1:172.21.15.84:6789/0 conn(0x7f056c0d3710 legacy=0x7f056c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056b7fe700 1 -- 172.21.15.84:0/613741065 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0b5150 con 0x7f056c0cfc60 2023-07-06T21:40:47.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f0569ffb700 1 -- 172.21.15.84:0/613741065 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574860325 0 0) 0x7f056c0d0a80 con 0x7f056c0d3710 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f056b7fe700 1 --2- 172.21.15.84:0/613741065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0d0f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f056c0cf520 tx=0x7f056400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006521b9dd88302 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f0569ffb700 1 -- 172.21.15.84:0/613741065 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056400e070 con 0x7f056c0cfc60 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f0569ffb700 1 -- 172.21.15.84:0/613741065 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056400bb20 con 0x7f056c0cfc60 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.176+0000 7f0569ffb700 1 -- 172.21.15.84:0/613741065 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564012430 con 0x7f056c0cfc60 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 -- 172.21.15.84:0/613741065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 msgr2=0x7f056c0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 --2- 172.21.15.84:0/613741065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0d0f10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f056c0cf520 tx=0x7f056400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 -- 172.21.15.84:0/613741065 shutdown_connections 2023-07-06T21:40:47.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 --2- 172.21.15.84:0/613741065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0d0f10 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 --2- 172.21.15.84:0/613741065 >> v2:172.21.15.107:3300/0 conn(0x7f056c0b83c0 0x7f056c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 -- 172.21.15.84:0/613741065 >> 172.21.15.84:0/613741065 conn(0x7f056c06b350 msgr2=0x7f056c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:47.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 -- 172.21.15.84:0/613741065 shutdown_connections 2023-07-06T21:40:47.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.177+0000 7f05719ff700 1 -- 172.21.15.84:0/613741065 wait complete. 2023-07-06T21:40:47.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.179+0000 7f05719ff700 1 Processor -- start 2023-07-06T21:40:47.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.180+0000 7f05719ff700 1 -- start start 2023-07-06T21:40:47.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f05719ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056c0b83c0 0x7f056c0c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f05719ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f05719ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056c0d3710 0x7f056c0b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f05719ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c156290 con 0x7f056c0cfc60 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f05719ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c156530 con 0x7f056c0d3710 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f05719ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f056c156810 con 0x7f056c0b83c0 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f056a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f056a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0b6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54512/0 (socket says 172.21.15.84:54512) 2023-07-06T21:40:47.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f056a7fc700 1 -- 172.21.15.84:0/48568343 learned_addr learned my addr 172.21.15.84:0/48568343 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:47.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f056b7fe700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056c0d3710 0x7f056c0b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.181+0000 7f056affd700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056c0b83c0 0x7f056c0c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f056a7fc700 1 -- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056c0b83c0 msgr2=0x7f056c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f056a7fc700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056c0b83c0 0x7f056c0c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f056a7fc700 1 -- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056c0d3710 msgr2=0x7f056c0b74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f056a7fc700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056c0d3710 0x7f056c0b74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f056a7fc700 1 -- 172.21.15.84:0/48568343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009a70 con 0x7f056c0cfc60 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f056a7fc700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0b6f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f056000c540 tx=0x7f056000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:47.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560004c80 con 0x7f056c0cfc60 2023-07-06T21:40:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0560004de0 con 0x7f056c0cfc60 2023-07-06T21:40:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c156af0 con 0x7f056c0cfc60 2023-07-06T21:40:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0560007310 con 0x7f056c0cfc60 2023-07-06T21:40:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.182+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c0c78c0 con 0x7f056c0cfc60 2023-07-06T21:40:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.183+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c156ce0 con 0x7f056c0cfc60 2023-07-06T21:40:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.184+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0560007470 con 0x7f056c0cfc60 2023-07-06T21:40:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.185+0000 7f0553fff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f055806ace0 0x7f055806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.185+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0560088ee0 con 0x7f056c0cfc60 2023-07-06T21:40:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.185+0000 7f056affd700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f055806ace0 0x7f055806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.186+0000 7f056affd700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f055806ace0 0x7f055806d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f055c009ad0 tx=0x7f055c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:47.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.190+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f05600542d0 con 0x7f056c0cfc60 2023-07-06T21:40:47.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.461+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f056c15b2c0 con 0x7f056c0cfc60 2023-07-06T21:40:47.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.465+0000 7f0553fff700 1 -- 172.21.15.84:0/48568343 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f05600578f0 con 0x7f056c0cfc60 2023-07-06T21:40:47.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f055806ace0 msgr2=0x7f055806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f055806ace0 0x7f055806d190 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f055c009ad0 tx=0x7f055c008040 comp rx=0 tx=0).stop 2023-07-06T21:40:47.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 msgr2=0x7f056c0b6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0b6f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f056000c540 tx=0x7f056000c900 comp rx=0 tx=0).stop 2023-07-06T21:40:47.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 shutdown_connections 2023-07-06T21:40:47.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f055806ace0 0x7f055806d190 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f056c0d3710 0x7f056c0b74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f056c0b83c0 0x7f056c0c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 --2- 172.21.15.84:0/48568343 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f056c0cfc60 0x7f056c0b6f60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 >> 172.21.15.84:0/48568343 conn(0x7f056c06b350 msgr2=0x7f056c0b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:47.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 shutdown_connections 2023-07-06T21:40:47.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.468+0000 7f05719ff700 1 -- 172.21.15.84:0/48568343 wait complete. 2023-07-06T21:40:47.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-07-06T21:40:47.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.678+0000 7f2bc05ff700 1 Processor -- start 2023-07-06T21:40:47.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.679+0000 7f2bc05ff700 1 -- start start 2023-07-06T21:40:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bc05ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2bb8070230 0x7f2bb80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bc05ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb80c9c10 0x7f2bb80ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bc05ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb815d930 con 0x7f2bb8159130 2023-07-06T21:40:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bc05ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb8070bf0 con 0x7f2bb8070230 2023-07-06T21:40:47.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bc05ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb815cd20 con 0x7f2bb80c9c10 2023-07-06T21:40:47.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bbe39b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2bb8159130 0x7f2bb80c4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42334/0 (socket says 172.21.15.84:42334) 2023-07-06T21:40:47.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bbe39b700 1 -- 172.21.15.84:0/3934484780 learned_addr learned my addr 172.21.15.84:0/3934484780 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:47.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bbdb9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2bb8070230 0x7f2bb80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bbeb9c700 1 --2- 172.21.15.84:0/3934484780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb80c9c10 0x7f2bb80ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.680+0000 7f2bbd399700 1 -- 172.21.15.84:0/3934484780 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612595850 0 0) 0x7f2bb815d930 con 0x7f2bb8159130 2023-07-06T21:40:47.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbdb9a700 1 -- 172.21.15.84:0/3934484780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb80c9c10 msgr2=0x7f2bb80ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbdb9a700 1 --2- 172.21.15.84:0/3934484780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb80c9c10 0x7f2bb80ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbdb9a700 1 -- 172.21.15.84:0/3934484780 >> v1:172.21.15.84:6789/0 conn(0x7f2bb8159130 legacy=0x7f2bb80c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbdb9a700 1 -- 172.21.15.84:0/3934484780 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb815d000 con 0x7f2bb8070230 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbdb9a700 1 --2- 172.21.15.84:0/3934484780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8070230 0x7f2bb80706b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009840 tx=0x7f2ba8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546f45165885d936 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbd399700 1 -- 172.21.15.84:0/3934484780 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8011830 con 0x7f2bb8070230 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbd399700 1 -- 172.21.15.84:0/3934484780 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8011990 con 0x7f2bb8070230 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.681+0000 7f2bbd399700 1 -- 172.21.15.84:0/3934484780 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8011d20 con 0x7f2bb8070230 2023-07-06T21:40:47.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3934484780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8070230 msgr2=0x7f2bb80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3934484780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8070230 0x7f2bb80706b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009840 tx=0x7f2ba8009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3934484780 shutdown_connections 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3934484780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb80c9c10 0x7f2bb80ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3934484780 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8070230 0x7f2bb80706b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3934484780 >> 172.21.15.84:0/3934484780 conn(0x7f2bb80cf3d0 msgr2=0x7f2bb80cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3934484780 shutdown_connections 2023-07-06T21:40:47.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.682+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3934484780 wait complete. 2023-07-06T21:40:47.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.684+0000 7f2bc05ff700 1 Processor -- start 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.684+0000 7f2bc05ff700 1 -- start start 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bc05ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb8070230 0x7f2bb806d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bc05ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb80c9c10 0x7f2bb8165df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bc05ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 0x7f2bb80ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bc05ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb815b3c0 con 0x7f2bb8159130 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bc05ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb815b610 con 0x7f2bb80c9c10 2023-07-06T21:40:47.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bc05ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb815b8f0 con 0x7f2bb8070230 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 0x7f2bb80ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 0x7f2bb80ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54532/0 (socket says 172.21.15.84:54532) 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 -- 172.21.15.84:0/3905224460 learned_addr learned my addr 172.21.15.84:0/3905224460 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbe39b700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb8070230 0x7f2bb806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbdb9a700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb80c9c10 0x7f2bb8165df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 -- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb8070230 msgr2=0x7f2bb806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb8070230 0x7f2bb806d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 -- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb80c9c10 msgr2=0x7f2bb8165df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb80c9c10 0x7f2bb8165df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.685+0000 7f2bbeb9c700 1 -- 172.21.15.84:0/3905224460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba80094a0 con 0x7f2bb8159130 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.686+0000 7f2bbeb9c700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 0x7f2bb80ab8a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0007e40 tx=0x7f2bb000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.686+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000ed30 con 0x7f2bb8159130 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.686+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb0014420 con 0x7f2bb8159130 2023-07-06T21:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.686+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb815bbd0 con 0x7f2bb8159130 2023-07-06T21:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.686+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00147d0 con 0x7f2bb8159130 2023-07-06T21:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.686+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb806d9b0 con 0x7f2bb8159130 2023-07-06T21:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.687+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8070630 con 0x7f2bb8159130 2023-07-06T21:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.688+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bb0014930 con 0x7f2bb8159130 2023-07-06T21:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.689+0000 7f2baeffd700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b9c06ace0 0x7f2b9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.689+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2bb0089740 con 0x7f2bb8159130 2023-07-06T21:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.689+0000 7f2bbe39b700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b9c06ace0 0x7f2b9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.690+0000 7f2bbe39b700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b9c06ace0 0x7f2b9c06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007fd0 tx=0x7f2bb4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.693+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2bb0054bd0 con 0x7f2bb8159130 2023-07-06T21:40:47.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:40:47.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:40:47.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.971+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f2bb8071400 con 0x7f2bb8159130 2023-07-06T21:40:47.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.975+0000 7f2baeffd700 1 -- 172.21.15.84:0/3905224460 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2bb00581f0 con 0x7f2bb8159130 2023-07-06T21:40:47.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b9c06ace0 msgr2=0x7f2b9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b9c06ace0 0x7f2b9c06d190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007fd0 tx=0x7f2bb4007de0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 msgr2=0x7f2bb80ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 0x7f2bb80ab8a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0007e40 tx=0x7f2bb000e3c0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 shutdown_connections 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2b9c06ace0 0x7f2b9c06d190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bb8159130 0x7f2bb80ab8a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bb8070230 0x7f2bb806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 --2- 172.21.15.84:0/3905224460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bb80c9c10 0x7f2bb8165df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:47.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 >> 172.21.15.84:0/3905224460 conn(0x7f2bb80cf3d0 msgr2=0x7f2bb8069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:47.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 shutdown_connections 2023-07-06T21:40:47.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:47.978+0000 7f2bc05ff700 1 -- 172.21.15.84:0/3905224460 wait complete. 2023-07-06T21:40:47.999 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-07-06T21:40:48.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.232+0000 7fabf31ff700 1 Processor -- start 2023-07-06T21:40:48.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.232+0000 7fabf31ff700 1 -- start start 2023-07-06T21:40:48.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.233+0000 7fabf31ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fabec15c6a0 0x7fabec06c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.233+0000 7fabf31ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec06b8a0 con 0x7fabec0b2470 2023-07-06T21:40:48.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.233+0000 7fabf31ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec06bb80 con 0x7fabec0b3010 2023-07-06T21:40:48.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.233+0000 7fabf31ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec06b6b0 con 0x7fabec15c6a0 2023-07-06T21:40:48.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabf179c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fabec0b2470 0x7fabec0cb040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42340/0 (socket says 172.21.15.84:42340) 2023-07-06T21:40:48.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabf179c700 1 -- 172.21.15.84:0/2302811326 learned_addr learned my addr 172.21.15.84:0/2302811326 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:48.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715293379 0 0) 0x7fabec06b8a0 con 0x7fabec0b2470 2023-07-06T21:40:48.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabdc0035b0 con 0x7fabec0b2470 2023-07-06T21:40:48.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabf0f9b700 1 --2- 172.21.15.84:0/2302811326 >> v2:172.21.15.107:3300/0 conn(0x7fabec15c6a0 0x7fabec06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013701864 0 0) 0x7fabdc0035b0 con 0x7fabec0b2470 2023-07-06T21:40:48.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 >> v2:172.21.15.107:3300/0 conn(0x7fabec15c6a0 msgr2=0x7fabec06c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:48.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 --2- 172.21.15.84:0/2302811326 >> v2:172.21.15.107:3300/0 conn(0x7fabec15c6a0 0x7fabec06c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 >> v1:172.21.15.107:6789/0 conn(0x7fabec0b3010 legacy=0x7fabec06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec06be60 con 0x7fabec0b2470 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.234+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fabe40019f0 con 0x7fabec0b2470 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 >> v1:172.21.15.84:6789/0 conn(0x7fabec0b2470 legacy=0x7fabec0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabdc003a00 0x7fabdc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabdc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabdc005200 0x7fabdc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00c050 con 0x7fabdc004850 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00c1c0 con 0x7fabdc003a00 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc00c380 con 0x7fabdc005200 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabebfff700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabdc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabf0f9b700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabdc003a00 0x7fabdc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabf179c700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabdc005200 0x7fabdc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabebfff700 1 -- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabdc005200 msgr2=0x7fabdc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:48.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabebfff700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabdc005200 0x7fabdc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabebfff700 1 -- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabdc003a00 msgr2=0x7fabdc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabebfff700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabdc003a00 0x7fabdc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabebfff700 1 -- 172.21.15.84:0/2302811326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0035b0 con 0x7fabdc004850 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.235+0000 7fabf179c700 1 -- 172.21.15.84:0/2302811326 reap_dead start 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabebfff700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabdc004cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fabd80017c0 tx=0x7fabdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581dbb9faf9c7d3c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd80052d0 con 0x7fabdc004850 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd8005430 con 0x7fabdc004850 2023-07-06T21:40:48.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabeb7fe700 1 -- 172.21.15.84:0/2302811326 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd80038c0 con 0x7fabdc004850 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 -- 172.21.15.84:0/2302811326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 msgr2=0x7fabdc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabdc004cc0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fabd80017c0 tx=0x7fabdc005880 comp rx=0 tx=0).stop 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 -- 172.21.15.84:0/2302811326 shutdown_connections 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 --2- 172.21.15.84:0/2302811326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabdc004cc0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 -- 172.21.15.84:0/2302811326 >> 172.21.15.84:0/2302811326 conn(0x7fabec06e640 msgr2=0x7fabec06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 -- 172.21.15.84:0/2302811326 shutdown_connections 2023-07-06T21:40:48.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.236+0000 7fabf31ff700 1 -- 172.21.15.84:0/2302811326 wait complete. 2023-07-06T21:40:48.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.238+0000 7fabf31ff700 1 Processor -- start 2023-07-06T21:40:48.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.238+0000 7fabf31ff700 1 -- start start 2023-07-06T21:40:48.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf31ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabec0b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf31ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabec1576d0 0x7fabec156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf31ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabec0c9520 0x7fabec15fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf31ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b3820 con 0x7fabdc004850 2023-07-06T21:40:48.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf31ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b3b00 con 0x7fabec0c9520 2023-07-06T21:40:48.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf31ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b3de0 con 0x7fabec1576d0 2023-07-06T21:40:48.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf0f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabec0b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf0f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabec0b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54544/0 (socket says 172.21.15.84:54544) 2023-07-06T21:40:48.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.239+0000 7fabf0f9b700 1 -- 172.21.15.84:0/1287164252 learned_addr learned my addr 172.21.15.84:0/1287164252 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:48.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabebfff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabec1576d0 0x7fabec156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf179c700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabec0c9520 0x7fabec15fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf0f9b700 1 -- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabec1576d0 msgr2=0x7fabec156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf0f9b700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabec1576d0 0x7fabec156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf0f9b700 1 -- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabec0c9520 msgr2=0x7fabec15fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf0f9b700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabec0c9520 0x7fabec15fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf0f9b700 1 -- 172.21.15.84:0/1287164252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0b40c0 con 0x7fabdc004850 2023-07-06T21:40:48.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf0f9b700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabec0b48c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fabe00119d0 tx=0x7fabe0011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:48.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe0009630 con 0x7fabdc004850 2023-07-06T21:40:48.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe0004240 con 0x7fabdc004850 2023-07-06T21:40:48.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec0b43a0 con 0x7fabdc004850 2023-07-06T21:40:48.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.240+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe000f730 con 0x7fabdc004850 2023-07-06T21:40:48.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.241+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec157200 con 0x7fabdc004850 2023-07-06T21:40:48.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.242+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec0709c0 con 0x7fabdc004850 2023-07-06T21:40:48.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.242+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fabe000f890 con 0x7fabdc004850 2023-07-06T21:40:48.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.244+0000 7fabe97fa700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fabd006ace0 0x7fabd006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.244+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fabe00897e0 con 0x7fabdc004850 2023-07-06T21:40:48.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.245+0000 7fabebfff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fabd006ace0 0x7fabd006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.245+0000 7fabebfff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fabd006ace0 0x7fabd006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabd8015040 tx=0x7fabd8009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:48.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.248+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fabe0054b40 con 0x7fabdc004850 2023-07-06T21:40:48.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.520+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fabec0c4080 con 0x7fabdc004850 2023-07-06T21:40:48.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.525+0000 7fabe97fa700 1 -- 172.21.15.84:0/1287164252 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7fabe0058160 con 0x7fabdc004850 2023-07-06T21:40:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fabd006ace0 msgr2=0x7fabd006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fabd006ace0 0x7fabd006d190 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabd8015040 tx=0x7fabd8009030 comp rx=0 tx=0).stop 2023-07-06T21:40:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 msgr2=0x7fabec0b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabec0b48c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fabe00119d0 tx=0x7fabe0011d90 comp rx=0 tx=0).stop 2023-07-06T21:40:48.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 shutdown_connections 2023-07-06T21:40:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fabd006ace0 0x7fabd006d190 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabec0c9520 0x7fabec15fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabdc004850 0x7fabec0b48c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 --2- 172.21.15.84:0/1287164252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fabec1576d0 0x7fabec156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 >> 172.21.15.84:0/1287164252 conn(0x7fabec06e640 msgr2=0x7fabec0ce5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 shutdown_connections 2023-07-06T21:40:48.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.527+0000 7fabf31ff700 1 -- 172.21.15.84:0/1287164252 wait complete. 2023-07-06T21:40:48.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-07-06T21:40:48.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.740+0000 7f3b231ff700 1 Processor -- start 2023-07-06T21:40:48.744 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.740+0000 7f3b231ff700 1 -- start start 2023-07-06T21:40:48.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b231ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3b1c0d3710 0x7f3b1c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b231ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b1c0cfa90 0x7f3b1c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b231ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c0cd910 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b231ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0cff10 con 0x7f3b1c0cfa90 2023-07-06T21:40:48.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b231ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0d0a80 con 0x7f3b1c0d3710 2023-07-06T21:40:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b2179c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b1c0cfa90 0x7f3b1c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b2179c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3b1c0cfa90 0x7f3b1c0d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54550/0 (socket says 172.21.15.84:54550) 2023-07-06T21:40:48.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.741+0000 7f3b2179c700 1 -- 172.21.15.84:0/3946823803 learned_addr learned my addr 172.21.15.84:0/3946823803 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b1bfff700 1 --2- 172.21.15.84:0/3946823803 >> v2:172.21.15.107:3300/0 conn(0x7f3b1c0d3710 0x7f3b1c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b2179c700 1 -- 172.21.15.84:0/3946823803 >> v2:172.21.15.107:3300/0 conn(0x7f3b1c0d3710 msgr2=0x7f3b1c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b2179c700 1 --2- 172.21.15.84:0/3946823803 >> v2:172.21.15.107:3300/0 conn(0x7f3b1c0d3710 0x7f3b1c0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b2179c700 1 -- 172.21.15.84:0/3946823803 >> v1:172.21.15.107:6789/0 conn(0x7f3b1c0b83c0 legacy=0x7f3b1c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b2179c700 1 -- 172.21.15.84:0/3946823803 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0b8790 con 0x7f3b1c0cfa90 2023-07-06T21:40:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b2179c700 1 --2- 172.21.15.84:0/3946823803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c0d0f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0b8cb0 tx=0x7f3b1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3656d8a1a5f5fe38 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b1b7fe700 1 -- 172.21.15.84:0/3946823803 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400e070 con 0x7f3b1c0cfa90 2023-07-06T21:40:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b1b7fe700 1 -- 172.21.15.84:0/3946823803 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14003a20 con 0x7f3b1c0cfa90 2023-07-06T21:40:48.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.742+0000 7f3b1b7fe700 1 -- 172.21.15.84:0/3946823803 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14003dd0 con 0x7f3b1c0cfa90 2023-07-06T21:40:48.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 -- 172.21.15.84:0/3946823803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0cfa90 msgr2=0x7f3b1c0d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 --2- 172.21.15.84:0/3946823803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c0d0f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0b8cb0 tx=0x7f3b1400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:48.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 -- 172.21.15.84:0/3946823803 shutdown_connections 2023-07-06T21:40:48.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 --2- 172.21.15.84:0/3946823803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c0d0f10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 --2- 172.21.15.84:0/3946823803 >> v2:172.21.15.107:3300/0 conn(0x7f3b1c0d3710 0x7f3b1c0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 -- 172.21.15.84:0/3946823803 >> 172.21.15.84:0/3946823803 conn(0x7f3b1c06b350 msgr2=0x7f3b1c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:48.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 -- 172.21.15.84:0/3946823803 shutdown_connections 2023-07-06T21:40:48.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.743+0000 7f3b231ff700 1 -- 172.21.15.84:0/3946823803 wait complete. 2023-07-06T21:40:48.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.744+0000 7f3b231ff700 1 Processor -- start 2023-07-06T21:40:48.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.745+0000 7f3b231ff700 1 -- start start 2023-07-06T21:40:48.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.745+0000 7f3b231ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 0x7f3b1c0c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.745+0000 7f3b231ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c1f1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b231ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b1c0d3710 0x7f3b1c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b231ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0c6670 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b231ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0c74d0 con 0x7f3b1c0cfa90 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b231ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0c7d50 con 0x7f3b1c0d3710 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 0x7f3b1c0c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 0x7f3b1c0c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54556/0 (socket says 172.21.15.84:54556) 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 -- 172.21.15.84:0/2723006637 learned_addr learned my addr 172.21.15.84:0/2723006637 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b1bfff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c1f1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b2179c700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b1c0d3710 0x7f3b1c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 -- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b1c0d3710 msgr2=0x7f3b1c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b1c0d3710 0x7f3b1c0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 -- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b1c0cfa90 msgr2=0x7f3b1c1f1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c1f1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 -- 172.21.15.84:0/2723006637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14009b30 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b1bfff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c1f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:48.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b2179c700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b1c0d3710 0x7f3b1c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:48.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.746+0000 7f3b20f9b700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 0x7f3b1c0c7810 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b4a0 tx=0x7f3b1000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:48.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.747+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10011830 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.747+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10011990 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.747+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c0c68c0 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.747+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10011d40 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.747+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c0c6e80 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.748+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c15a760 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.749+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3b1001a400 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.750+0000 7f3b197fa700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b0006ace0 0x7f3b0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:48.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.750+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3b10088e60 con 0x7f3b1c0b83c0 2023-07-06T21:40:48.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.750+0000 7f3b1bfff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b0006ace0 0x7f3b0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.751+0000 7f3b1bfff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b0006ace0 0x7f3b0006d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c151080 tx=0x7f3b08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:48.755+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3b10054260 con 0x7f3b1c0b83c0 2023-07-06T21:40:49.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.030+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f3b1c0b4100 con 0x7f3b1c0b83c0 2023-07-06T21:40:49.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.033+0000 7f3b197fa700 1 -- 172.21.15.84:0/2723006637 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3b100150a0 con 0x7f3b1c0b83c0 2023-07-06T21:40:49.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b0006ace0 msgr2=0x7f3b0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b0006ace0 0x7f3b0006d190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c151080 tx=0x7f3b08008040 comp rx=0 tx=0).stop 2023-07-06T21:40:49.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 msgr2=0x7f3b1c0c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 0x7f3b1c0c7810 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000b4a0 tx=0x7f3b1000b860 comp rx=0 tx=0).stop 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 shutdown_connections 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3b0006ace0 0x7f3b0006d190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b1c0d3710 0x7f3b1c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c0b83c0 0x7f3b1c0c7810 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 --2- 172.21.15.84:0/2723006637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b1c0cfa90 0x7f3b1c1f1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 >> 172.21.15.84:0/2723006637 conn(0x7f3b1c06b350 msgr2=0x7f3b1c15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:49.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 shutdown_connections 2023-07-06T21:40:49.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.036+0000 7f3b231ff700 1 -- 172.21.15.84:0/2723006637 wait complete. 2023-07-06T21:40:49.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.247+0000 7f612b9ff700 1 Processor -- start 2023-07-06T21:40:49.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.247+0000 7f612b9ff700 1 -- start start 2023-07-06T21:40:49.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.248+0000 7f612b9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61240d3710 0x7f61240cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.248+0000 7f612b9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240cfa90 0x7f61240d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.248+0000 7f612b9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61240cd910 con 0x7f61240b83c0 2023-07-06T21:40:49.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.248+0000 7f612b9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61240cff10 con 0x7f61240d3710 2023-07-06T21:40:49.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.248+0000 7f612b9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61240d0a80 con 0x7f61240cfa90 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61240d3710 0x7f61240cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61240d3710 0x7f61240cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54558/0 (socket says 172.21.15.84:54558) 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 -- 172.21.15.84:0/1112346802 learned_addr learned my addr 172.21.15.84:0/1112346802 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6129f9c700 1 --2- 172.21.15.84:0/1112346802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240cfa90 0x7f61240d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 -- 172.21.15.84:0/1112346802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240cfa90 msgr2=0x7f61240d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 --2- 172.21.15.84:0/1112346802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240cfa90 0x7f61240d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 -- 172.21.15.84:0/1112346802 >> v1:172.21.15.107:6789/0 conn(0x7f61240b83c0 legacy=0x7f61240cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 -- 172.21.15.84:0/1112346802 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61240b8790 con 0x7f61240d3710 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.249+0000 7f6128f9a700 1 --2- 172.21.15.84:0/1112346802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f61240cf550 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f61140098f0 tx=0x7f6114009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbe9e97fa425122 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f611bfff700 1 -- 172.21.15.84:0/1112346802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611400b8a0 con 0x7f61240d3710 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f611bfff700 1 -- 172.21.15.84:0/1112346802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611400ba00 con 0x7f61240d3710 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f611bfff700 1 -- 172.21.15.84:0/1112346802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611400bd90 con 0x7f61240d3710 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 -- 172.21.15.84:0/1112346802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 msgr2=0x7f61240cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 --2- 172.21.15.84:0/1112346802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f61240cf550 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f61140098f0 tx=0x7f6114009c00 comp rx=0 tx=0).stop 2023-07-06T21:40:49.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 -- 172.21.15.84:0/1112346802 shutdown_connections 2023-07-06T21:40:49.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 --2- 172.21.15.84:0/1112346802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240cfa90 0x7f61240d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 --2- 172.21.15.84:0/1112346802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f61240cf550 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 -- 172.21.15.84:0/1112346802 >> 172.21.15.84:0/1112346802 conn(0x7f612406b350 msgr2=0x7f612406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:49.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 -- 172.21.15.84:0/1112346802 shutdown_connections 2023-07-06T21:40:49.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.250+0000 7f612b9ff700 1 -- 172.21.15.84:0/1112346802 wait complete. 2023-07-06T21:40:49.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.253+0000 7f612b9ff700 1 Processor -- start 2023-07-06T21:40:49.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.253+0000 7f612b9ff700 1 -- start start 2023-07-06T21:40:49.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612b9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240b83c0 0x7f61241502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612b9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61240cfa90 0x7f6124150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612b9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f6124150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612b9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61240cc650 con 0x7f61240d3710 2023-07-06T21:40:49.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612b9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61240cc930 con 0x7f61240cfa90 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612b9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61240ca6c0 con 0x7f61240b83c0 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f6129f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f6124150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f6129f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f6124150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54564/0 (socket says 172.21.15.84:54564) 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f6129f9c700 1 -- 172.21.15.84:0/2042033728 learned_addr learned my addr 172.21.15.84:0/2042033728 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f6128f9a700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61240cfa90 0x7f6124150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.254+0000 7f612979b700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240b83c0 0x7f61241502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6129f9c700 1 -- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240b83c0 msgr2=0x7f61241502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6129f9c700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240b83c0 0x7f61241502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6129f9c700 1 -- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61240cfa90 msgr2=0x7f6124150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6129f9c700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61240cfa90 0x7f6124150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6129f9c700 1 -- 172.21.15.84:0/2042033728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114009550 con 0x7f61240d3710 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f612979b700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240b83c0 0x7f61241502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6129f9c700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f6124150ea0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f611c007e60 tx=0x7f611c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c00ede0 con 0x7f61240d3710 2023-07-06T21:40:49.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c014420 con 0x7f61240d3710 2023-07-06T21:40:49.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61240ca9a0 con 0x7f61240d3710 2023-07-06T21:40:49.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0147d0 con 0x7f61240d3710 2023-07-06T21:40:49.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.255+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61240cc290 con 0x7f61240d3710 2023-07-06T21:40:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.256+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61240cc420 con 0x7f61240d3710 2023-07-06T21:40:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.257+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f611c014930 con 0x7f61240d3710 2023-07-06T21:40:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.258+0000 7f6119ffb700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f610806ace0 0x7f610806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.258+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f611c0898a0 con 0x7f61240d3710 2023-07-06T21:40:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.259+0000 7f612979b700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f610806ace0 0x7f610806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.259+0000 7f612979b700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f610806ace0 0x7f610806d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6120000f60 tx=0x7f6120009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:49.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.263+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f611c054c00 con 0x7f61240d3710 2023-07-06T21:40:49.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.460+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61240cc790 con 0x7f61240d3710 2023-07-06T21:40:49.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.461+0000 7f6119ffb700 1 -- 172.21.15.84:0/2042033728 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14639 (secure 0 0 0) 0x7f61240cc790 con 0x7f61240d3710 2023-07-06T21:40:49.464 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:49.464 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":33,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:45.399981+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:49.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f610806ace0 msgr2=0x7f610806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f610806ace0 0x7f610806d190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6120000f60 tx=0x7f6120009040 comp rx=0 tx=0).stop 2023-07-06T21:40:49.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 msgr2=0x7f6124150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f6124150ea0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f611c007e60 tx=0x7f611c00e3c0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 shutdown_connections 2023-07-06T21:40:49.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f610806ace0 0x7f610806d190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61240d3710 0x7f6124150ea0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61240b83c0 0x7f61241502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.464+0000 7f612b9ff700 1 --2- 172.21.15.84:0/2042033728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61240cfa90 0x7f6124150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.465+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 >> 172.21.15.84:0/2042033728 conn(0x7f612406b350 msgr2=0x7f61240b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.465+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 shutdown_connections 2023-07-06T21:40:49.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.465+0000 7f612b9ff700 1 -- 172.21.15.84:0/2042033728 wait complete. 2023-07-06T21:40:49.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:40:49.486 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:40:49.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.682+0000 7f01121ff700 1 Processor -- start 2023-07-06T21:40:49.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.682+0000 7f01121ff700 1 -- start start 2023-07-06T21:40:49.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f01121ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f010c15c020 0x7f010c0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f01121ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f010c15c460 0x7f010c069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f01121ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c900 0x7f010c06a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f01121ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c15d400 con 0x7f010c15c460 2023-07-06T21:40:49.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f01121ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f010c06cf40 con 0x7f010c15c020 2023-07-06T21:40:49.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f01121ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c15ce40 con 0x7f010c15c900 2023-07-06T21:40:49.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f010c15c460 0x7f010c069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f010c15c460 0x7f010c069e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54578/0 (socket says 172.21.15.84:54578) 2023-07-06T21:40:49.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 -- 172.21.15.84:0/4102712491 learned_addr learned my addr 172.21.15.84:0/4102712491 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:49.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f010b7fe700 1 --2- 172.21.15.84:0/4102712491 >> v2:172.21.15.107:3300/0 conn(0x7f010c15c020 0x7f010c0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f010bfff700 1 --2- 172.21.15.84:0/4102712491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c900 0x7f010c06a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 -- 172.21.15.84:0/4102712491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c900 msgr2=0x7f010c06a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:49.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 --2- 172.21.15.84:0/4102712491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c900 0x7f010c06a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 -- 172.21.15.84:0/4102712491 >> v2:172.21.15.107:3300/0 conn(0x7f010c15c020 msgr2=0x7f010c0c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.684+0000 7f0103fff700 1 --2- 172.21.15.84:0/4102712491 >> v2:172.21.15.107:3300/0 conn(0x7f010c15c020 0x7f010c0c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f0103fff700 1 -- 172.21.15.84:0/4102712491 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c15d120 con 0x7f010c15c460 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f0103fff700 1 --2- 172.21.15.84:0/4102712491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c069e90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f010c0cf850 tx=0x7f00f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77d37407d7ab323 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f010affd700 1 -- 172.21.15.84:0/4102712491 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4011830 con 0x7f010c15c460 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f010affd700 1 -- 172.21.15.84:0/4102712491 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f4011990 con 0x7f010c15c460 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f010affd700 1 -- 172.21.15.84:0/4102712491 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4011d20 con 0x7f010c15c460 2023-07-06T21:40:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f01121ff700 1 -- 172.21.15.84:0/4102712491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 msgr2=0x7f010c069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.685+0000 7f01121ff700 1 --2- 172.21.15.84:0/4102712491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c069e90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f010c0cf850 tx=0x7f00f4009bd0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 -- 172.21.15.84:0/4102712491 shutdown_connections 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 --2- 172.21.15.84:0/4102712491 >> v2:172.21.15.107:3300/0 conn(0x7f010c15c020 0x7f010c0c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 --2- 172.21.15.84:0/4102712491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c900 0x7f010c06a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 --2- 172.21.15.84:0/4102712491 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c069e90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 -- 172.21.15.84:0/4102712491 >> 172.21.15.84:0/4102712491 conn(0x7f010c0cbf10 msgr2=0x7f010c0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 -- 172.21.15.84:0/4102712491 shutdown_connections 2023-07-06T21:40:49.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.686+0000 7f01121ff700 1 -- 172.21.15.84:0/4102712491 wait complete. 2023-07-06T21:40:49.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.688+0000 7f01121ff700 1 Processor -- start 2023-07-06T21:40:49.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.688+0000 7f01121ff700 1 -- start start 2023-07-06T21:40:49.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f01121ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c020 0x7f010c140c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f01121ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c0c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f01121ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f010c15c900 0x7f010c0c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f01121ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c13fbf0 con 0x7f010c15c460 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f01121ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c13fed0 con 0x7f010c15c900 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f01121ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010c1401b0 con 0x7f010c15c020 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c0c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c0c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54592/0 (socket says 172.21.15.84:54592) 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 -- 172.21.15.84:0/2939532296 learned_addr learned my addr 172.21.15.84:0/2939532296 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f010b7fe700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c020 0x7f010c140c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f010bfff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f010c15c900 0x7f010c0c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 -- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c020 msgr2=0x7f010c140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c020 0x7f010c140c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 -- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f010c15c900 msgr2=0x7f010c0c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f010c15c900 0x7f010c0c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f0103fff700 1 -- 172.21.15.84:0/2939532296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009550 con 0x7f010c15c460 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.689+0000 7f010bfff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f010c15c900 0x7f010c0c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.690+0000 7f0103fff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c0c2180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009c10 tx=0x7f00f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:49.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.690+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004510 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.690+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f401e070 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.690+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c140490 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.690+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f40038e0 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.690+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c13f710 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.691+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c1409b0 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.692+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00f4003a40 con 0x7f010c15c460 2023-07-06T21:40:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.693+0000 7f0108ff9700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00ec06ace0 0x7f00ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.693+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f00f4089ff0 con 0x7f010c15c460 2023-07-06T21:40:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.693+0000 7f010b7fe700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00ec06ace0 0x7f00ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.694+0000 7f010b7fe700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00ec06ace0 0x7f00ec06d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009a50 tx=0x7f00fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:49.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.697+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00f40bc310 con 0x7f010c15c460 2023-07-06T21:40:49.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.891+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f010c04e1d0 con 0x7f010c15c460 2023-07-06T21:40:49.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.891+0000 7f0108ff9700 1 -- 172.21.15.84:0/2939532296 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00f40150a0 con 0x7f010c15c460 2023-07-06T21:40:49.893 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:40:49.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.893+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00ec06ace0 msgr2=0x7f00ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.893+0000 7f01121ff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00ec06ace0 0x7f00ec06d190 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009a50 tx=0x7f00fc008040 comp rx=0 tx=0).stop 2023-07-06T21:40:49.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.893+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 msgr2=0x7f010c0c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:49.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.893+0000 7f01121ff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c0c2180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009c10 tx=0x7f00f4004750 comp rx=0 tx=0).stop 2023-07-06T21:40:49.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 shutdown_connections 2023-07-06T21:40:49.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00ec06ace0 0x7f00ec06d190 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f010c15c900 0x7f010c0c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f010c15c020 0x7f010c140c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 --2- 172.21.15.84:0/2939532296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f010c15c460 0x7f010c0c2180 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:49.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 >> 172.21.15.84:0/2939532296 conn(0x7f010c0cbf10 msgr2=0x7f010c0702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:49.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 shutdown_connections 2023-07-06T21:40:49.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:49.894+0000 7f01121ff700 1 -- 172.21.15.84:0/2939532296 wait complete. 2023-07-06T21:40:49.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:40:50.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.108+0000 7f8d889ff700 1 Processor -- start 2023-07-06T21:40:50.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.108+0000 7f8d889ff700 1 -- start start 2023-07-06T21:40:50.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.109+0000 7f8d889ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d840b81c0 0x7f8d840cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.109+0000 7f8d889ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8d840ceda0 0x7f8d840cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.109+0000 7f8d889ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840d1830 con 0x7f8d840b1a80 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.109+0000 7f8d889ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840cf330 con 0x7f8d840b81c0 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.109+0000 7f8d889ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840d1270 con 0x7f8d840ceda0 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d81d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8d840b1a80 0x7f8d8415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42350/0 (socket says 172.21.15.84:42350) 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d81d9b700 1 -- 172.21.15.84:0/2062887408 learned_addr learned my addr 172.21.15.84:0/2062887408 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:50.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8259c700 1 --2- 172.21.15.84:0/2062887408 >> v2:172.21.15.84:3300/0 conn(0x7f8d840b81c0 0x7f8d840cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d82d9d700 1 --2- 172.21.15.84:0/2062887408 >> v2:172.21.15.107:3300/0 conn(0x7f8d840ceda0 0x7f8d840cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8159a700 1 -- 172.21.15.84:0/2062887408 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79743064 0 0) 0x7f8d840d1830 con 0x7f8d840b1a80 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8159a700 1 -- 172.21.15.84:0/2062887408 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d6c0035b0 con 0x7f8d840b1a80 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8259c700 1 -- 172.21.15.84:0/2062887408 >> v2:172.21.15.107:3300/0 conn(0x7f8d840ceda0 msgr2=0x7f8d840cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8259c700 1 --2- 172.21.15.84:0/2062887408 >> v2:172.21.15.107:3300/0 conn(0x7f8d840ceda0 0x7f8d840cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8259c700 1 -- 172.21.15.84:0/2062887408 >> v1:172.21.15.84:6789/0 conn(0x7f8d840b1a80 legacy=0x7f8d8415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.110+0000 7f8d8259c700 1 -- 172.21.15.84:0/2062887408 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840d1550 con 0x7f8d840b81c0 2023-07-06T21:40:50.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d8259c700 1 --2- 172.21.15.84:0/2062887408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d840cf600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8d840c9c50 tx=0x7f8d74009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aaa680968b4e4d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d8159a700 1 -- 172.21.15.84:0/2062887408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74011830 con 0x7f8d840b81c0 2023-07-06T21:40:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d8159a700 1 -- 172.21.15.84:0/2062887408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74011990 con 0x7f8d840b81c0 2023-07-06T21:40:50.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d8159a700 1 -- 172.21.15.84:0/2062887408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74011d20 con 0x7f8d840b81c0 2023-07-06T21:40:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 -- 172.21.15.84:0/2062887408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 msgr2=0x7f8d840cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 --2- 172.21.15.84:0/2062887408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d840cf600 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8d840c9c50 tx=0x7f8d74009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 -- 172.21.15.84:0/2062887408 shutdown_connections 2023-07-06T21:40:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 --2- 172.21.15.84:0/2062887408 >> v2:172.21.15.107:3300/0 conn(0x7f8d840ceda0 0x7f8d840cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 --2- 172.21.15.84:0/2062887408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d840cf600 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 -- 172.21.15.84:0/2062887408 >> 172.21.15.84:0/2062887408 conn(0x7f8d8406b2b0 msgr2=0x7f8d8406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 -- 172.21.15.84:0/2062887408 shutdown_connections 2023-07-06T21:40:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.111+0000 7f8d889ff700 1 -- 172.21.15.84:0/2062887408 wait complete. 2023-07-06T21:40:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.113+0000 7f8d889ff700 1 Processor -- start 2023-07-06T21:40:50.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.113+0000 7f8d889ff700 1 -- start start 2023-07-06T21:40:50.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d889ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d840b1a80 0x7f8d84159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d889ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d8415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d889ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d840ceda0 0x7f8d841e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d889ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841e27b0 con 0x7f8d840b81c0 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d889ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841e1c30 con 0x7f8d840ceda0 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d889ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d841e1f10 con 0x7f8d840b1a80 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d81d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d8415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d81d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d8415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54610/0 (socket says 172.21.15.84:54610) 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.114+0000 7f8d81d9b700 1 -- 172.21.15.84:0/611345021 learned_addr learned my addr 172.21.15.84:0/611345021 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d8259c700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d840b1a80 0x7f8d84159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d82d9d700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d840ceda0 0x7f8d841e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d81d9b700 1 -- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d840b1a80 msgr2=0x7f8d84159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d81d9b700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d840b1a80 0x7f8d84159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d81d9b700 1 -- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d840ceda0 msgr2=0x7f8d841e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d81d9b700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d840ceda0 0x7f8d841e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d81d9b700 1 -- 172.21.15.84:0/611345021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d740094a0 con 0x7f8d840b81c0 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d8259c700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d840b1a80 0x7f8d84159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d81d9b700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d8415a4b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8d78001eb0 tx=0x7f8d7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78005d70 con 0x7f8d840b81c0 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d780070d0 con 0x7f8d840b81c0 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d841e21f0 con 0x7f8d840b81c0 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d78007460 con 0x7f8d840b81c0 2023-07-06T21:40:50.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.115+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8415b210 con 0x7f8d840b81c0 2023-07-06T21:40:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.116+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d840c67b0 con 0x7f8d840b81c0 2023-07-06T21:40:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.117+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d780075c0 con 0x7f8d840b81c0 2023-07-06T21:40:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.118+0000 7f8d72ffd700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d6406ace0 0x7f8d6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.118+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8d78088700 con 0x7f8d840b81c0 2023-07-06T21:40:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.118+0000 7f8d8259c700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d6406ace0 0x7f8d6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.119+0000 7f8d8259c700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d6406ace0 0x7f8d6406d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d740090b0 tx=0x7f8d74008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.123+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d840c67b0 con 0x7f8d840b81c0 2023-07-06T21:40:50.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.320+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8d840c67b0 con 0x7f8d840b81c0 2023-07-06T21:40:50.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.321+0000 7f8d72ffd700 1 -- 172.21.15.84:0/611345021 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8d840c67b0 con 0x7f8d840b81c0 2023-07-06T21:40:50.323 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:40:50.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d6406ace0 msgr2=0x7f8d6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d6406ace0 0x7f8d6406d190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8d740090b0 tx=0x7f8d74008030 comp rx=0 tx=0).stop 2023-07-06T21:40:50.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 msgr2=0x7f8d8415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d8415a4b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8d78001eb0 tx=0x7f8d7800c860 comp rx=0 tx=0).stop 2023-07-06T21:40:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 shutdown_connections 2023-07-06T21:40:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d6406ace0 0x7f8d6406d190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d840ceda0 0x7f8d841e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d840b1a80 0x7f8d84159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 --2- 172.21.15.84:0/611345021 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d840b81c0 0x7f8d8415a4b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 >> 172.21.15.84:0/611345021 conn(0x7f8d8406b2b0 msgr2=0x7f8d840b70b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:50.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 shutdown_connections 2023-07-06T21:40:50.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.323+0000 7f8d889ff700 1 -- 172.21.15.84:0/611345021 wait complete. 2023-07-06T21:40:50.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T21:40:50.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.534+0000 7f5c281ff700 1 Processor -- start 2023-07-06T21:40:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.534+0000 7f5c281ff700 1 -- start start 2023-07-06T21:40:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.535+0000 7f5c281ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d3580 0x7f5c200d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.535+0000 7f5c281ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200d0990 con 0x7f5c200d13b0 2023-07-06T21:40:50.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.535+0000 7f5c281ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c200d1830 con 0x7f5c200b8530 2023-07-06T21:40:50.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.535+0000 7f5c281ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200d3af0 con 0x7f5c200d3580 2023-07-06T21:40:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c2679c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5c200d13b0 0x7f5c2015c8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42364/0 (socket says 172.21.15.84:42364) 2023-07-06T21:40:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c2679c700 1 -- 172.21.15.84:0/1485648807 learned_addr learned my addr 172.21.15.84:0/1485648807 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c2579a700 1 --2- 172.21.15.84:0/1485648807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d3580 0x7f5c200d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345376960 0 0) 0x7f5c200d0990 con 0x7f5c200d13b0 2023-07-06T21:40:50.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c0035b0 con 0x7f5c200d13b0 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203830928 0 0) 0x7f5c200d1830 con 0x7f5c200b8530 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c200d0990 con 0x7f5c200b8530 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c2579a700 1 -- 172.21.15.84:0/1485648807 >> v1:172.21.15.107:6789/0 conn(0x7f5c200b8530 legacy=0x7f5c200cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c2579a700 1 -- 172.21.15.84:0/1485648807 >> v1:172.21.15.84:6789/0 conn(0x7f5c200d13b0 legacy=0x7f5c2015c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.536+0000 7f5c2579a700 1 -- 172.21.15.84:0/1485648807 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200cd9d0 con 0x7f5c200d3580 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.537+0000 7f5c2579a700 1 --2- 172.21.15.84:0/1485648807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d3580 0x7f5c200d0e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009870 tx=0x7f5c10009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5800501b994e411 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.537+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011830 con 0x7f5c200d3580 2023-07-06T21:40:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.537+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c10011990 con 0x7f5c200d3580 2023-07-06T21:40:50.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.537+0000 7f5c24f99700 1 -- 172.21.15.84:0/1485648807 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011d20 con 0x7f5c200d3580 2023-07-06T21:40:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.537+0000 7f5c281ff700 1 -- 172.21.15.84:0/1485648807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d3580 msgr2=0x7f5c200d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.537+0000 7f5c281ff700 1 --2- 172.21.15.84:0/1485648807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d3580 0x7f5c200d0e70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009870 tx=0x7f5c10009b80 comp rx=0 tx=0).stop 2023-07-06T21:40:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.538+0000 7f5c281ff700 1 -- 172.21.15.84:0/1485648807 shutdown_connections 2023-07-06T21:40:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.538+0000 7f5c281ff700 1 --2- 172.21.15.84:0/1485648807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d3580 0x7f5c200d0e70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.538+0000 7f5c281ff700 1 -- 172.21.15.84:0/1485648807 >> 172.21.15.84:0/1485648807 conn(0x7f5c200715c0 msgr2=0x7f5c20071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:50.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.538+0000 7f5c281ff700 1 -- 172.21.15.84:0/1485648807 shutdown_connections 2023-07-06T21:40:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.538+0000 7f5c281ff700 1 -- 172.21.15.84:0/1485648807 wait complete. 2023-07-06T21:40:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.539+0000 7f5c281ff700 1 Processor -- start 2023-07-06T21:40:50.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.539+0000 7f5c281ff700 1 -- start start 2023-07-06T21:40:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.540+0000 7f5c281ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 0x7f5c201517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.540+0000 7f5c281ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d13b0 0x7f5c200cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.540+0000 7f5c281ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c200d3580 0x7f5c2015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.540+0000 7f5c281ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c20155b30 con 0x7f5c200b8530 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.540+0000 7f5c281ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c67f0 con 0x7f5c200d3580 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.540+0000 7f5c281ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200cc390 con 0x7f5c200d13b0 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 0x7f5c201517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 0x7f5c201517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54624/0 (socket says 172.21.15.84:54624) 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 -- 172.21.15.84:0/2561227474 learned_addr learned my addr 172.21.15.84:0/2561227474 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 -- 172.21.15.84:0/2561227474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d13b0 msgr2=0x7f5c200cba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d13b0 0x7f5c200cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 -- 172.21.15.84:0/2561227474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c200d3580 msgr2=0x7f5c2015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c200d3580 0x7f5c2015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 -- 172.21.15.84:0/2561227474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100094a0 con 0x7f5c200b8530 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c25f9b700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 0x7f5c201517c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c700 tx=0x7f5c1c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0052d0 con 0x7f5c200b8530 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1c005430 con 0x7f5c200b8530 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.541+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c200cc670 con 0x7f5c200b8530 2023-07-06T21:40:50.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.542+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c1c0038f0 con 0x7f5c200b8530 2023-07-06T21:40:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.542+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c20151d00 con 0x7f5c200b8530 2023-07-06T21:40:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.543+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c200c6ca0 con 0x7f5c200b8530 2023-07-06T21:40:50.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.544+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5c1c003a50 con 0x7f5c200b8530 2023-07-06T21:40:50.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.544+0000 7f5c167fc700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5c0406ace0 0x7f5c0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.545+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5c1c088890 con 0x7f5c200b8530 2023-07-06T21:40:50.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.545+0000 7f5c2579a700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5c0406ace0 0x7f5c0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.545+0000 7f5c2579a700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5c0406ace0 0x7f5c0406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c201582e0 tx=0x7f5c10010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.549+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5c200c6ca0 con 0x7f5c200b8530 2023-07-06T21:40:50.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.746+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5c200b4f80 con 0x7f5c200b8530 2023-07-06T21:40:50.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.746+0000 7f5c167fc700 1 -- 172.21.15.84:0/2561227474 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5c1c017020 con 0x7f5c200b8530 2023-07-06T21:40:50.749 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:50.749 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-07-06T21:40:50.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:50.750 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-07-06T21:40:50.750 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-07-06T21:40:50.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:40:50.750 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:40:50.751 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2023-07-06T21:40:50.751 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2023-07-06T21:40:50.751 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:40:50.751 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-07-06T21:40:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5c0406ace0 msgr2=0x7f5c0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5c0406ace0 0x7f5c0406d190 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c201582e0 tx=0x7f5c10010000 comp rx=0 tx=0).stop 2023-07-06T21:40:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 msgr2=0x7f5c201517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 0x7f5c201517c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c00c700 tx=0x7f5c1c00cac0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 shutdown_connections 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5c0406ace0 0x7f5c0406d190 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c200d3580 0x7f5c2015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5c200b8530 0x7f5c201517c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 --2- 172.21.15.84:0/2561227474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c200d13b0 0x7f5c200cba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 >> 172.21.15.84:0/2561227474 conn(0x7f5c200715c0 msgr2=0x7f5c200cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 shutdown_connections 2023-07-06T21:40:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.749+0000 7f5c281ff700 1 -- 172.21.15.84:0/2561227474 wait complete. 2023-07-06T21:40:50.769 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T21:40:50.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T21:40:50.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.967+0000 7f3f005ff700 1 Processor -- start 2023-07-06T21:40:50.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.967+0000 7f3f005ff700 1 -- start start 2023-07-06T21:40:50.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3f005ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3ef815fee0 0x7f3ef81602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3f005ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef80ceef0 con 0x7f3ef8160ac0 2023-07-06T21:40:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3f005ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef80cf450 con 0x7f3ef80d0ea0 2023-07-06T21:40:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3f005ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef8160820 con 0x7f3ef815fee0 2023-07-06T21:40:50.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3efdb9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3ef8160ac0 0x7f3ef80cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42380/0 (socket says 172.21.15.84:42380) 2023-07-06T21:40:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3efdb9a700 1 -- 172.21.15.84:0/3478123233 learned_addr learned my addr 172.21.15.84:0/3478123233 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.968+0000 7f3efeb9c700 1 --2- 172.21.15.84:0/3478123233 >> v2:172.21.15.84:3300/0 conn(0x7f3ef815fee0 0x7f3ef81602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107821479 0 0) 0x7f3ef80ceef0 con 0x7f3ef8160ac0 2023-07-06T21:40:50.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee40035b0 con 0x7f3ef8160ac0 2023-07-06T21:40:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946256661 0 0) 0x7f3ef80cf450 con 0x7f3ef80d0ea0 2023-07-06T21:40:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ef80ceef0 con 0x7f3ef80d0ea0 2023-07-06T21:40:50.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efeb9c700 1 -- 172.21.15.84:0/3478123233 >> v1:172.21.15.107:6789/0 conn(0x7f3ef80d0ea0 legacy=0x7f3ef80d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efeb9c700 1 -- 172.21.15.84:0/3478123233 >> v1:172.21.15.84:6789/0 conn(0x7f3ef8160ac0 legacy=0x7f3ef80cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efeb9c700 1 -- 172.21.15.84:0/3478123233 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80cf120 con 0x7f3ef815fee0 2023-07-06T21:40:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efeb9c700 1 --2- 172.21.15.84:0/3478123233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef815fee0 0x7f3ef81602e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009d90 tx=0x7f3ef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998ad0efeb0d9e6a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000e070 con 0x7f3ef815fee0 2023-07-06T21:40:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef000bb20 con 0x7f3ef815fee0 2023-07-06T21:40:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.969+0000 7f3efd399700 1 -- 172.21.15.84:0/3478123233 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0012430 con 0x7f3ef815fee0 2023-07-06T21:40:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 -- 172.21.15.84:0/3478123233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef815fee0 msgr2=0x7f3ef81602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 --2- 172.21.15.84:0/3478123233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef815fee0 0x7f3ef81602e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009d90 tx=0x7f3ef000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 -- 172.21.15.84:0/3478123233 shutdown_connections 2023-07-06T21:40:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 --2- 172.21.15.84:0/3478123233 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef815fee0 0x7f3ef81602e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 -- 172.21.15.84:0/3478123233 >> 172.21.15.84:0/3478123233 conn(0x7f3ef80715c0 msgr2=0x7f3ef8071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 -- 172.21.15.84:0/3478123233 shutdown_connections 2023-07-06T21:40:50.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.970+0000 7f3f005ff700 1 -- 172.21.15.84:0/3478123233 wait complete. 2023-07-06T21:40:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.972+0000 7f3f005ff700 1 Processor -- start 2023-07-06T21:40:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.972+0000 7f3f005ff700 1 -- start start 2023-07-06T21:40:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3f005ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ef815fee0 0x7f3ef80b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3f005ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 0x7f3ef80c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3f005ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef80ca2b0 0x7f3ef80cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3f005ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80b7f10 con 0x7f3ef8160ac0 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3f005ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80ca750 con 0x7f3ef80ca2b0 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3f005ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80b7060 con 0x7f3ef815fee0 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 0x7f3ef80c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 0x7f3ef80c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54644/0 (socket says 172.21.15.84:54644) 2023-07-06T21:40:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 -- 172.21.15.84:0/1275143541 learned_addr learned my addr 172.21.15.84:0/1275143541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:50.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efeb9c700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef80ca2b0 0x7f3ef80cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efe39b700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ef815fee0 0x7f3ef80b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 -- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ef815fee0 msgr2=0x7f3ef80b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ef815fee0 0x7f3ef80b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 -- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef80ca2b0 msgr2=0x7f3ef80cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:50.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef80ca2b0 0x7f3ef80cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:50.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.973+0000 7f3efdb9a700 1 -- 172.21.15.84:0/1275143541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0009a70 con 0x7f3ef8160ac0 2023-07-06T21:40:50.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.974+0000 7f3efdb9a700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 0x7f3ef80c9d70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8001eb0 tx=0x7f3ee800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.974+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8005d70 con 0x7f3ef8160ac0 2023-07-06T21:40:50.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.974+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee80070d0 con 0x7f3ef8160ac0 2023-07-06T21:40:50.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.974+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef80b7290 con 0x7f3ef8160ac0 2023-07-06T21:40:50.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.974+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8007460 con 0x7f3ef8160ac0 2023-07-06T21:40:50.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.974+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef80b7850 con 0x7f3ef8160ac0 2023-07-06T21:40:50.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.975+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef80b8050 con 0x7f3ef8160ac0 2023-07-06T21:40:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.976+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ee80075c0 con 0x7f3ef8160ac0 2023-07-06T21:40:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.977+0000 7f3eeeffd700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3edc06ad10 0x7f3edc06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.977+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3ee8088720 con 0x7f3ef8160ac0 2023-07-06T21:40:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.977+0000 7f3efe39b700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3edc06ad10 0x7f3edc06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.978+0000 7f3efe39b700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3edc06ad10 0x7f3edc06d1c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001230 tx=0x7f3ef4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:50.981+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3ef80b8050 con 0x7f3ef8160ac0 2023-07-06T21:40:51.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:51.175+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3ef80b8050 con 0x7f3ef8160ac0 2023-07-06T21:40:52.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.164+0000 7f3eeeffd700 1 -- 172.21.15.84:0/1275143541 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3ef80b8050 con 0x7f3ef8160ac0 2023-07-06T21:40:52.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3edc06ad10 msgr2=0x7f3edc06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:52.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3edc06ad10 0x7f3edc06d1c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4001230 tx=0x7f3ef4009040 comp rx=0 tx=0).stop 2023-07-06T21:40:52.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 msgr2=0x7f3ef80c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 0x7f3ef80c9d70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8001eb0 tx=0x7f3ee800c860 comp rx=0 tx=0).stop 2023-07-06T21:40:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 shutdown_connections 2023-07-06T21:40:52.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3edc06ad10 0x7f3edc06d1c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ef80ca2b0 0x7f3ef80cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ef815fee0 0x7f3ef80b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 --2- 172.21.15.84:0/1275143541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef8160ac0 0x7f3ef80c9d70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.170+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 >> 172.21.15.84:0/1275143541 conn(0x7f3ef80715c0 msgr2=0x7f3ef815c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.171+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 shutdown_connections 2023-07-06T21:40:52.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.171+0000 7f3f005ff700 1 -- 172.21.15.84:0/1275143541 wait complete. 2023-07-06T21:40:52.175 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-07-06T21:40:52.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T21:40:52.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.395+0000 7f3c5b3ff700 1 Processor -- start 2023-07-06T21:40:52.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.396+0000 7f3c5b3ff700 1 -- start start 2023-07-06T21:40:52.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5b3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:52.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5b3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c5406b6b0 0x7f3c540ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:52.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5b3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5415cd30 con 0x7f3c540cfef0 2023-07-06T21:40:52.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5b3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540d1ad0 con 0x7f3c5406b6b0 2023-07-06T21:40:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5b3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540d1640 con 0x7f3c5415c960 2023-07-06T21:40:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5999c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c5406b6b0 0x7f3c540ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5899a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3c540cfef0 0x7f3c5406afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42386/0 (socket says 172.21.15.84:42386) 2023-07-06T21:40:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5899a700 1 -- 172.21.15.84:0/2475140830 learned_addr learned my addr 172.21.15.84:0/2475140830 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:52.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5919b700 1 --2- 172.21.15.84:0/2475140830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:52.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5999c700 1 -- 172.21.15.84:0/2475140830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 msgr2=0x7f3c540cdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5999c700 1 --2- 172.21.15.84:0/2475140830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540cdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c5999c700 1 -- 172.21.15.84:0/2475140830 >> v1:172.21.15.84:6789/0 conn(0x7f3c540cfef0 legacy=0x7f3c5406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c5999c700 1 -- 172.21.15.84:0/2475140830 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540d0a30 con 0x7f3c5406b6b0 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.397+0000 7f3c4bfff700 1 -- 172.21.15.84:0/2475140830 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 422536066 0 0) 0x7f3c44001110 con 0x7f3c540cfef0 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c5999c700 1 --2- 172.21.15.84:0/2475140830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5406b6b0 0x7f3c540ce6e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009d90 tx=0x7f3c4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853cc1c71abb2cbc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c4bfff700 1 -- 172.21.15.84:0/2475140830 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00e070 con 0x7f3c5406b6b0 2023-07-06T21:40:52.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c4bfff700 1 -- 172.21.15.84:0/2475140830 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c00bb20 con 0x7f3c5406b6b0 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c4bfff700 1 -- 172.21.15.84:0/2475140830 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c012430 con 0x7f3c5406b6b0 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/2475140830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5406b6b0 msgr2=0x7f3c540ce6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.398+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/2475140830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5406b6b0 0x7f3c540ce6e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009d90 tx=0x7f3c4c00b330 comp rx=0 tx=0).stop 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.399+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/2475140830 shutdown_connections 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.399+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/2475140830 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c5406b6b0 0x7f3c540ce6e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.399+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/2475140830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540cdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.399+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/2475140830 >> 172.21.15.84:0/2475140830 conn(0x7f3c5406a7b0 msgr2=0x7f3c54099de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:52.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.399+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/2475140830 shutdown_connections 2023-07-06T21:40:52.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.399+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/2475140830 wait complete. 2023-07-06T21:40:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.401+0000 7f3c5b3ff700 1 Processor -- start 2023-07-06T21:40:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.401+0000 7f3c5b3ff700 1 -- start start 2023-07-06T21:40:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.402+0000 7f3c5b3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c5406b6b0 0x7f3c540c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.402+0000 7f3c5b3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 0x7f3c5414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.402+0000 7f3c5b3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:52.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5b3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540ccee0 con 0x7f3c540cfef0 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5b3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c540c4ab0 con 0x7f3c5406b6b0 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5b3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5414d390 con 0x7f3c5415c960 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 0x7f3c5414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 0x7f3c5414ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54654/0 (socket says 172.21.15.84:54654) 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 -- 172.21.15.84:0/3858169202 learned_addr learned my addr 172.21.15.84:0/3858169202 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5999c700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:52.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5919b700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c5406b6b0 0x7f3c540c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:52.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 -- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 msgr2=0x7f3c540ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:52.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 -- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c5406b6b0 msgr2=0x7f3c540c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c5406b6b0 0x7f3c540c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 -- 172.21.15.84:0/3858169202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009a70 con 0x7f3c540cfef0 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.403+0000 7f3c5899a700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 0x7f3c5414ce50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c550 tx=0x7f3c4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.404+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44004c80 con 0x7f3c540cfef0 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.404+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c44004e00 con 0x7f3c540cfef0 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.404+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c540caab0 con 0x7f3c540cfef0 2023-07-06T21:40:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.404+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c44007310 con 0x7f3c540cfef0 2023-07-06T21:40:52.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.404+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c540cc5b0 con 0x7f3c540cfef0 2023-07-06T21:40:52.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.405+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c540cd070 con 0x7f3c540cfef0 2023-07-06T21:40:52.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.406+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c44007470 con 0x7f3c540cfef0 2023-07-06T21:40:52.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.407+0000 7f3c49ffb700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c3806ace0 0x7f3c3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:52.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.408+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3c44088df0 con 0x7f3c540cfef0 2023-07-06T21:40:52.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.408+0000 7f3c5919b700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c3806ace0 0x7f3c3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:52.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.408+0000 7f3c5919b700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c3806ace0 0x7f3c3806d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009ad0 tx=0x7f3c50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:52.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.412+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c44054080 con 0x7f3c540cfef0 2023-07-06T21:40:52.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:52.606+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3c5415cd30 con 0x7f3c540cfef0 2023-07-06T21:40:53.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.168+0000 7f3c49ffb700 1 -- 172.21.15.84:0/3858169202 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3c5415cd30 con 0x7f3c540cfef0 2023-07-06T21:40:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.175+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c3806ace0 msgr2=0x7f3c3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.175+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c3806ace0 0x7f3c3806d190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c50009ad0 tx=0x7f3c50008040 comp rx=0 tx=0).stop 2023-07-06T21:40:53.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.175+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 msgr2=0x7f3c5414ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.175+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 0x7f3c5414ce50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3c4400c550 tx=0x7f3c4400c860 comp rx=0 tx=0).stop 2023-07-06T21:40:53.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.177+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 shutdown_connections 2023-07-06T21:40:53.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.177+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c3806ace0 0x7f3c3806d190 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.177+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c5415c960 0x7f3c540ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.177+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c5406b6b0 0x7f3c540c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.177+0000 7f3c5b3ff700 1 --2- 172.21.15.84:0/3858169202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c540cfef0 0x7f3c5414ce50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.177+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 >> 172.21.15.84:0/3858169202 conn(0x7f3c5406a7b0 msgr2=0x7f3c540ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:53.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.181+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 shutdown_connections 2023-07-06T21:40:53.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.182+0000 7f3c5b3ff700 1 -- 172.21.15.84:0/3858169202 wait complete. 2023-07-06T21:40:53.186 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-07-06T21:40:53.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T21:40:53.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.401+0000 7febf9fff700 1 Processor -- start 2023-07-06T21:40:53.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.401+0000 7febf9fff700 1 -- start start 2023-07-06T21:40:53.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf9fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7febf415c020 0x7febf40c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf9fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf406cf40 con 0x7febf415c460 2023-07-06T21:40:53.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf9fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf415ce40 con 0x7febf406a5a0 2023-07-06T21:40:53.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf9fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf415d400 con 0x7febf415c020 2023-07-06T21:40:53.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febebfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7febf415c460 0x7febf4069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48230/0 (socket says 172.21.15.84:48230) 2023-07-06T21:40:53.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febebfff700 1 -- 172.21.15.84:0/1979985601 learned_addr learned my addr 172.21.15.84:0/1979985601 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:53.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf37fe700 1 --2- 172.21.15.84:0/1979985601 >> v2:172.21.15.84:3300/0 conn(0x7febf415c020 0x7febf40c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf37fe700 1 -- 172.21.15.84:0/1979985601 >> v1:172.21.15.107:6789/0 conn(0x7febf406a5a0 legacy=0x7febf406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf37fe700 1 -- 172.21.15.84:0/1979985601 >> v1:172.21.15.84:6789/0 conn(0x7febf415c460 legacy=0x7febf4069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf37fe700 1 -- 172.21.15.84:0/1979985601 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf415d120 con 0x7febf415c020 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.403+0000 7febf2ffd700 1 -- 172.21.15.84:0/1979985601 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1075809388 0 0) 0x7febf406cf40 con 0x7febf415c460 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.404+0000 7febf37fe700 1 --2- 172.21.15.84:0/1979985601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c020 0x7febf40c90a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7febe4009840 tx=0x7febe4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2127dba1fae9c67d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.404+0000 7febf2ffd700 1 -- 172.21.15.84:0/1979985601 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011830 con 0x7febf415c020 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.404+0000 7febf2ffd700 1 -- 172.21.15.84:0/1979985601 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe4011990 con 0x7febf415c020 2023-07-06T21:40:53.407 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.404+0000 7febf2ffd700 1 -- 172.21.15.84:0/1979985601 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4011d20 con 0x7febf415c020 2023-07-06T21:40:53.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.404+0000 7febf9fff700 1 -- 172.21.15.84:0/1979985601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c020 msgr2=0x7febf40c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.404+0000 7febf9fff700 1 --2- 172.21.15.84:0/1979985601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c020 0x7febf40c90a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7febe4009840 tx=0x7febe4009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:53.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.405+0000 7febf9fff700 1 -- 172.21.15.84:0/1979985601 shutdown_connections 2023-07-06T21:40:53.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.405+0000 7febf9fff700 1 --2- 172.21.15.84:0/1979985601 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c020 0x7febf40c90a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.405+0000 7febf9fff700 1 -- 172.21.15.84:0/1979985601 >> 172.21.15.84:0/1979985601 conn(0x7febf40cbf10 msgr2=0x7febf40ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:53.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.405+0000 7febf9fff700 1 -- 172.21.15.84:0/1979985601 shutdown_connections 2023-07-06T21:40:53.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.405+0000 7febf9fff700 1 -- 172.21.15.84:0/1979985601 wait complete. 2023-07-06T21:40:53.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.406+0000 7febf9fff700 1 Processor -- start 2023-07-06T21:40:53.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.406+0000 7febf9fff700 1 -- start start 2023-07-06T21:40:53.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.407+0000 7febf9fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febf406a5a0 0x7febf40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.407+0000 7febf9fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 0x7febf40c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.407+0000 7febf9fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c460 0x7febf414bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.407+0000 7febf9fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf414a3f0 con 0x7febf415c460 2023-07-06T21:40:53.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.407+0000 7febf9fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b28b0 con 0x7febf415c020 2023-07-06T21:40:53.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.407+0000 7febf9fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40b2b90 con 0x7febf406a5a0 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c460 0x7febf414bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 0x7febf40c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf37fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febf406a5a0 0x7febf40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febebfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 0x7febf40c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:55682/0 (socket says 172.21.15.84:55682) 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febebfff700 1 -- 172.21.15.84:0/1398122827 learned_addr learned my addr 172.21.15.84:0/1398122827 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 -- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febf406a5a0 msgr2=0x7febf40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febf406a5a0 0x7febf40b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 -- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 msgr2=0x7febf40c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 0x7febf40c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 -- 172.21.15.84:0/1398122827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe40094a0 con 0x7febf415c460 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febebfff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 0x7febf40c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf37fe700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febf406a5a0 0x7febf40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.408+0000 7febf3fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c460 0x7febf414bc40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7febec00cb10 tx=0x7febec00ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.409+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec003c00 con 0x7febf415c460 2023-07-06T21:40:53.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.409+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec003d60 con 0x7febf415c460 2023-07-06T21:40:53.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.409+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00a9c0 con 0x7febf415c460 2023-07-06T21:40:53.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.409+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf40b2e70 con 0x7febf415c460 2023-07-06T21:40:53.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.409+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf40b3430 con 0x7febf415c460 2023-07-06T21:40:53.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.410+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf40b3650 con 0x7febf415c460 2023-07-06T21:40:53.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.411+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7febec00ab20 con 0x7febf415c460 2023-07-06T21:40:53.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.412+0000 7febf0ff9700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7febd406ace0 0x7febd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.415 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.412+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5962+0+0 (secure 0 0 0) 0x7febec089d10 con 0x7febf415c460 2023-07-06T21:40:53.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.412+0000 7febf37fe700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7febd406ace0 0x7febd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.416 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.413+0000 7febf37fe700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7febd406ace0 0x7febd406d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7febe40090b0 tx=0x7febe4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:53.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.416+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7febec054e90 con 0x7febf415c460 2023-07-06T21:40:53.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.656+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7febf404e1d0 con 0x7febf415c460 2023-07-06T21:40:53.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.664+0000 7febf0ff9700 1 -- 172.21.15.84:0/1398122827 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7febec0584b0 con 0x7febf415c460 2023-07-06T21:40:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.669+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7febd406ace0 msgr2=0x7febd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.669+0000 7febf9fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7febd406ace0 0x7febd406d190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7febe40090b0 tx=0x7febe4008030 comp rx=0 tx=0).stop 2023-07-06T21:40:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.669+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c460 msgr2=0x7febf414bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.669+0000 7febf9fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c460 0x7febf414bc40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7febec00cb10 tx=0x7febec00ce20 comp rx=0 tx=0).stop 2023-07-06T21:40:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 shutdown_connections 2023-07-06T21:40:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7febd406ace0 0x7febd406d190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7febf415c460 0x7febf414bc40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febf406a5a0 0x7febf40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 --2- 172.21.15.84:0/1398122827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febf415c020 0x7febf40c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 >> 172.21.15.84:0/1398122827 conn(0x7febf40cbf10 msgr2=0x7febf40ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.670+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 shutdown_connections 2023-07-06T21:40:53.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.672+0000 7febf9fff700 1 -- 172.21.15.84:0/1398122827 wait complete. 2023-07-06T21:40:53.681 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 5 and data pool 6 2023-07-06T21:40:53.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:53.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.897+0000 7f8354dff700 1 Processor -- start 2023-07-06T21:40:53.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.897+0000 7f8354dff700 1 -- start start 2023-07-06T21:40:53.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.898+0000 7f8354dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500b83c0 0x7f83500cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.898+0000 7f8354dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83500cfc60 0x7f83500d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.898+0000 7f8354dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83500d0a80 con 0x7f83500d3710 2023-07-06T21:40:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.898+0000 7f8354dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83500b8790 con 0x7f83500cfc60 2023-07-06T21:40:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.898+0000 7f8354dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83500b4e70 con 0x7f83500b83c0 2023-07-06T21:40:53.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83500cfc60 0x7f83500d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834dd9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f83500d3710 0x7f83500cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48246/0 (socket says 172.21.15.84:48246) 2023-07-06T21:40:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834dd9b700 1 -- 172.21.15.84:0/291961218 learned_addr learned my addr 172.21.15.84:0/291961218 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834e59c700 1 --2- 172.21.15.84:0/291961218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500b83c0 0x7f83500cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834ed9d700 1 -- 172.21.15.84:0/291961218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500b83c0 msgr2=0x7f83500cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834ed9d700 1 --2- 172.21.15.84:0/291961218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500b83c0 0x7f83500cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834ed9d700 1 -- 172.21.15.84:0/291961218 >> v1:172.21.15.84:6789/0 conn(0x7f83500d3710 legacy=0x7f83500cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834ed9d700 1 -- 172.21.15.84:0/291961218 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83500b5150 con 0x7f83500cfc60 2023-07-06T21:40:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834ed9d700 1 --2- 172.21.15.84:0/291961218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500cfc60 0x7f83500d0f10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8348009d90 tx=0x7f834800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e2fb57a63b2650 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:53.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.899+0000 7f834d59a700 1 -- 172.21.15.84:0/291961218 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800e070 con 0x7f83500cfc60 2023-07-06T21:40:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f834d59a700 1 -- 172.21.15.84:0/291961218 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800bb20 con 0x7f83500cfc60 2023-07-06T21:40:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f834d59a700 1 -- 172.21.15.84:0/291961218 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8348012430 con 0x7f83500cfc60 2023-07-06T21:40:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 -- 172.21.15.84:0/291961218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500cfc60 msgr2=0x7f83500d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 --2- 172.21.15.84:0/291961218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500cfc60 0x7f83500d0f10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8348009d90 tx=0x7f834800b330 comp rx=0 tx=0).stop 2023-07-06T21:40:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 -- 172.21.15.84:0/291961218 shutdown_connections 2023-07-06T21:40:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 --2- 172.21.15.84:0/291961218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500cfc60 0x7f83500d0f10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 --2- 172.21.15.84:0/291961218 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500b83c0 0x7f83500cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 -- 172.21.15.84:0/291961218 >> 172.21.15.84:0/291961218 conn(0x7f835006b350 msgr2=0x7f835006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 -- 172.21.15.84:0/291961218 shutdown_connections 2023-07-06T21:40:53.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.900+0000 7f8354dff700 1 -- 172.21.15.84:0/291961218 wait complete. 2023-07-06T21:40:53.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.902+0000 7f8354dff700 1 Processor -- start 2023-07-06T21:40:53.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.902+0000 7f8354dff700 1 -- start start 2023-07-06T21:40:53.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f8354dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 0x7f83501552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f8354dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500cfc60 0x7f835014acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f8354dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500d3710 0x7f83500cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f8354dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83500c8dd0 con 0x7f83500b83c0 2023-07-06T21:40:53.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f8354dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83500c8f60 con 0x7f83500d3710 2023-07-06T21:40:53.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f8354dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83500c9490 con 0x7f83500cfc60 2023-07-06T21:40:53.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f834e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 0x7f83501552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f834e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 0x7f83501552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53156/0 (socket says 172.21.15.84:53156) 2023-07-06T21:40:53.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f834e59c700 1 -- 172.21.15.84:0/2995517294 learned_addr learned my addr 172.21.15.84:0/2995517294 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:53.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f834dd9b700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500cfc60 0x7f835014acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.903+0000 7f834ed9d700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500d3710 0x7f83500cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f834e59c700 1 -- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500cfc60 msgr2=0x7f835014acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f834e59c700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500cfc60 0x7f835014acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f834e59c700 1 -- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500d3710 msgr2=0x7f83500cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:53.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f834e59c700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500d3710 0x7f83500cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:53.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f834e59c700 1 -- 172.21.15.84:0/2995517294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348009a70 con 0x7f83500b83c0 2023-07-06T21:40:53.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f834e59c700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 0x7f83501552c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f834000e8a0 tx=0x7f834000ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:53.916 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8340009590 con 0x7f83500b83c0 2023-07-06T21:40:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8340013680 con 0x7f83500b83c0 2023-07-06T21:40:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83500c91b0 con 0x7f83500b83c0 2023-07-06T21:40:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.904+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83400139f0 con 0x7f83500b83c0 2023-07-06T21:40:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.905+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83500c7490 con 0x7f83500b83c0 2023-07-06T21:40:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.906+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83500cd700 con 0x7f83500b83c0 2023-07-06T21:40:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.907+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8340013b50 con 0x7f83500b83c0 2023-07-06T21:40:53.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.908+0000 7f833effd700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f833006ace0 0x7f833006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:53.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.908+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83400892b0 con 0x7f83500b83c0 2023-07-06T21:40:53.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.908+0000 7f834dd9b700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f833006ace0 0x7f833006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:53.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.909+0000 7f834dd9b700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f833006ace0 0x7f833006d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83500cb190 tx=0x7f8344009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:53.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:53.913+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8340057de0 con 0x7f83500b83c0 2023-07-06T21:40:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.115+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83500b2150 con 0x7f83500b83c0 2023-07-06T21:40:54.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.117+0000 7f833effd700 1 -- 172.21.15.84:0/2995517294 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18364 (secure 0 0 0) 0x7f8340057970 con 0x7f83500b83c0 2023-07-06T21:40:54.119 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:54.119 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":36,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:53.657242+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f833006ace0 msgr2=0x7f833006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f833006ace0 0x7f833006d190 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83500cb190 tx=0x7f8344009040 comp rx=0 tx=0).stop 2023-07-06T21:40:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 msgr2=0x7f83501552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 0x7f83501552c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f834000e8a0 tx=0x7f834000ebb0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 shutdown_connections 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f833006ace0 0x7f833006d190 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83500d3710 0x7f83500cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83500b83c0 0x7f83501552c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 --2- 172.21.15.84:0/2995517294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83500cfc60 0x7f835014acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 >> 172.21.15.84:0/2995517294 conn(0x7f835006b350 msgr2=0x7f83500b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 shutdown_connections 2023-07-06T21:40:54.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.119+0000 7f8354dff700 1 -- 172.21.15.84:0/2995517294 wait complete. 2023-07-06T21:40:54.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:54.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.333+0000 7f69803ff700 1 Processor -- start 2023-07-06T21:40:54.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.333+0000 7f69803ff700 1 -- start start 2023-07-06T21:40:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f69803ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f69780c99d0 0x7f697806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f69803ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697806a820 con 0x7f6978069e90 2023-07-06T21:40:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f69803ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69780c3f80 con 0x7f6978158830 2023-07-06T21:40:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f69803ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69780c4260 con 0x7f69780c99d0 2023-07-06T21:40:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f697d99a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6978069e90 0x7f69780c92c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48254/0 (socket says 172.21.15.84:48254) 2023-07-06T21:40:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f697d99a700 1 -- 172.21.15.84:0/3830144800 learned_addr learned my addr 172.21.15.84:0/3830144800 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f697e99c700 1 --2- 172.21.15.84:0/3830144800 >> v2:172.21.15.84:3300/0 conn(0x7f69780c99d0 0x7f697806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f697d199700 1 -- 172.21.15.84:0/3830144800 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12683629 0 0) 0x7f6968001110 con 0x7f6978069e90 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f697d199700 1 -- 172.21.15.84:0/3830144800 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69640034d0 con 0x7f6978069e90 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.335+0000 7f697e99c700 1 -- 172.21.15.84:0/3830144800 >> v1:172.21.15.107:6789/0 conn(0x7f6978158830 legacy=0x7f69780c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.336+0000 7f697e99c700 1 -- 172.21.15.84:0/3830144800 >> v1:172.21.15.84:6789/0 conn(0x7f6978069e90 legacy=0x7f69780c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.336+0000 7f697e99c700 1 -- 172.21.15.84:0/3830144800 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69780c4540 con 0x7f69780c99d0 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.336+0000 7f697e99c700 1 --2- 172.21.15.84:0/3830144800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69780c99d0 0x7f697806a2e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6970009d90 tx=0x7f697000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51882a59bf09a63f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.336+0000 7f697d199700 1 -- 172.21.15.84:0/3830144800 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697000e070 con 0x7f69780c99d0 2023-07-06T21:40:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.336+0000 7f697d199700 1 -- 172.21.15.84:0/3830144800 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697000bb20 con 0x7f69780c99d0 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.336+0000 7f697d199700 1 -- 172.21.15.84:0/3830144800 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970012430 con 0x7f69780c99d0 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 -- 172.21.15.84:0/3830144800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69780c99d0 msgr2=0x7f697806a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 --2- 172.21.15.84:0/3830144800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69780c99d0 0x7f697806a2e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6970009d90 tx=0x7f697000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 -- 172.21.15.84:0/3830144800 shutdown_connections 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 --2- 172.21.15.84:0/3830144800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69780c99d0 0x7f697806a2e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 -- 172.21.15.84:0/3830144800 >> 172.21.15.84:0/3830144800 conn(0x7f69780cf3d0 msgr2=0x7f69780cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 -- 172.21.15.84:0/3830144800 shutdown_connections 2023-07-06T21:40:54.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.337+0000 7f69803ff700 1 -- 172.21.15.84:0/3830144800 wait complete. 2023-07-06T21:40:54.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.338+0000 7f69803ff700 1 Processor -- start 2023-07-06T21:40:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 -- start start 2023-07-06T21:40:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 0x7f697814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69780c99d0 0x7f69781514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6978158830 0x7f6978151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6978159bd0 con 0x7f6978069e90 2023-07-06T21:40:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6978159eb0 con 0x7f69780c99d0 2023-07-06T21:40:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.339+0000 7f69803ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697815a190 con 0x7f6978158830 2023-07-06T21:40:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 0x7f697814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 0x7f697814a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53182/0 (socket says 172.21.15.84:53182) 2023-07-06T21:40:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 -- 172.21.15.84:0/3980396272 learned_addr learned my addr 172.21.15.84:0/3980396272 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 -- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6978158830 msgr2=0x7f6978151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6978158830 0x7f6978151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 -- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69780c99d0 msgr2=0x7f69781514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697d99a700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69780c99d0 0x7f69781514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69780c99d0 0x7f69781514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 -- 172.21.15.84:0/3980396272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6970009a70 con 0x7f6978069e90 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697d99a700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69780c99d0 0x7f69781514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f697e19b700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 0x7f697814a5b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f697400c450 tx=0x7f697400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.340+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740052d0 con 0x7f6978069e90 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.341+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974005430 con 0x7f6978069e90 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.341+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697815a470 con 0x7f6978069e90 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.341+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69740038c0 con 0x7f6978069e90 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.341+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697814aaf0 con 0x7f6978069e90 2023-07-06T21:40:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.342+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69780ada90 con 0x7f6978069e90 2023-07-06T21:40:54.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.343+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6974003a20 con 0x7f6978069e90 2023-07-06T21:40:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.344+0000 7f696effd700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f695c06ad10 0x7f695c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.344+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f69740888f0 con 0x7f6978069e90 2023-07-06T21:40:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.344+0000 7f697d99a700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f695c06ad10 0x7f695c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.345+0000 7f697d99a700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f695c06ad10 0x7f695c06d1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6968007fd0 tx=0x7f6968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:54.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.349+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6974053990 con 0x7f6978069e90 2023-07-06T21:40:54.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.547+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69780c39b0 con 0x7f6978069e90 2023-07-06T21:40:54.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.548+0000 7f696effd700 1 -- 172.21.15.84:0/3980396272 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18364 (secure 0 0 0) 0x7f6974056fb0 con 0x7f6978069e90 2023-07-06T21:40:54.550 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:54.551 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":36,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:53.657242+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:54.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f695c06ad10 msgr2=0x7f695c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f695c06ad10 0x7f695c06d1c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6968007fd0 tx=0x7f6968009040 comp rx=0 tx=0).stop 2023-07-06T21:40:54.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 msgr2=0x7f697814a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 0x7f697814a5b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f697400c450 tx=0x7f697400c810 comp rx=0 tx=0).stop 2023-07-06T21:40:54.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 shutdown_connections 2023-07-06T21:40:54.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f695c06ad10 0x7f695c06d1c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6978158830 0x7f6978151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6978069e90 0x7f697814a5b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 --2- 172.21.15.84:0/3980396272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69780c99d0 0x7f69781514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 >> 172.21.15.84:0/3980396272 conn(0x7f69780cf3d0 msgr2=0x7f6978153530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:54.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 shutdown_connections 2023-07-06T21:40:54.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.551+0000 7f69803ff700 1 -- 172.21.15.84:0/3980396272 wait complete. 2023-07-06T21:40:54.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T21:40:54.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.767+0000 7ff56e3ff700 1 Processor -- start 2023-07-06T21:40:54.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.768+0000 7ff56e3ff700 1 -- start start 2023-07-06T21:40:54.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56e3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff5680b81c0 0x7ff5680cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56e3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5680d1830 con 0x7ff5680ceda0 2023-07-06T21:40:54.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56e3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5680cf330 con 0x7ff5680b81c0 2023-07-06T21:40:54.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5680d1270 con 0x7ff5680b1a80 2023-07-06T21:40:54.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff567fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff5680b81c0 0x7ff5680cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56c99c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7ff5680ceda0 0x7ff56815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:43668/0 (socket says 172.21.15.84:43668) 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.769+0000 7ff56c99c700 1 -- 172.21.15.84:0/1160418048 learned_addr learned my addr 172.21.15.84:0/1160418048 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff566ffd700 1 -- 172.21.15.84:0/1160418048 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457817896 0 0) 0x7ff5680d1830 con 0x7ff5680ceda0 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff566ffd700 1 -- 172.21.15.84:0/1160418048 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5500035b0 con 0x7ff5680ceda0 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff567fff700 1 -- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 msgr2=0x7ff5680cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff567fff700 1 --2- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff567fff700 1 -- 172.21.15.84:0/1160418048 >> v1:172.21.15.107:6789/0 conn(0x7ff5680ceda0 legacy=0x7ff56815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff567fff700 1 -- 172.21.15.84:0/1160418048 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680d1550 con 0x7ff5680b81c0 2023-07-06T21:40:54.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff567fff700 1 --2- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680b81c0 0x7ff5680cf600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff558009840 tx=0x7ff558009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e265f0b19bb1ee0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:54.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff566ffd700 1 -- 172.21.15.84:0/1160418048 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011830 con 0x7ff5680b81c0 2023-07-06T21:40:54.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff566ffd700 1 -- 172.21.15.84:0/1160418048 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff558011990 con 0x7ff5680b81c0 2023-07-06T21:40:54.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.770+0000 7ff566ffd700 1 -- 172.21.15.84:0/1160418048 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011d20 con 0x7ff5680b81c0 2023-07-06T21:40:54.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 -- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680b81c0 msgr2=0x7ff5680cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680b81c0 0x7ff5680cf600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff558009840 tx=0x7ff558009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:54.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 -- 172.21.15.84:0/1160418048 shutdown_connections 2023-07-06T21:40:54.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680b81c0 0x7ff5680cf600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/1160418048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 -- 172.21.15.84:0/1160418048 >> 172.21.15.84:0/1160418048 conn(0x7ff56806b2b0 msgr2=0x7ff56806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:54.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 -- 172.21.15.84:0/1160418048 shutdown_connections 2023-07-06T21:40:54.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.771+0000 7ff56e3ff700 1 -- 172.21.15.84:0/1160418048 wait complete. 2023-07-06T21:40:54.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.773+0000 7ff56e3ff700 1 Processor -- start 2023-07-06T21:40:54.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.774+0000 7ff56e3ff700 1 -- start start 2023-07-06T21:40:54.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56e3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 0x7ff5680cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56e3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680ceda0 0x7ff5681f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56e3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681f55f0 con 0x7ff5680b81c0 2023-07-06T21:40:54.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56e3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681f5890 con 0x7ff5680ceda0 2023-07-06T21:40:54.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681f5b70 con 0x7ff5680b1a80 2023-07-06T21:40:54.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 0x7ff5680cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 0x7ff5680cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53186/0 (socket says 172.21.15.84:53186) 2023-07-06T21:40:54.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 -- 172.21.15.84:0/2613707395 learned_addr learned my addr 172.21.15.84:0/2613707395 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff567fff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56c99c700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680ceda0 0x7ff5681f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 -- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 msgr2=0x7ff5680c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 -- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680ceda0 msgr2=0x7ff5681f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680ceda0 0x7ff5681f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:54.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff5677fe700 1 -- 172.21.15.84:0/2613707395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5580094a0 con 0x7ff5680b81c0 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.775+0000 7ff56c99c700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680ceda0 0x7ff5681f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.776+0000 7ff5677fe700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 0x7ff5680cc100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e9b0 tx=0x7ff55c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.776+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c009710 con 0x7ff5680b81c0 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.776+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c00c850 con 0x7ff5680b81c0 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.776+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c00cbc0 con 0x7ff5680b81c0 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.776+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5681f5e50 con 0x7ff5680b81c0 2023-07-06T21:40:54.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.776+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5680c6c60 con 0x7ff5680b81c0 2023-07-06T21:40:54.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.777+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56804e1d0 con 0x7ff5680b81c0 2023-07-06T21:40:54.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.778+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff55c00cd20 con 0x7ff5680b81c0 2023-07-06T21:40:54.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.779+0000 7ff564ff9700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff55406ace0 0x7ff55406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:54.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.779+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff55c089590 con 0x7ff5680b81c0 2023-07-06T21:40:54.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.779+0000 7ff567fff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff55406ace0 0x7ff55406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:54.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.780+0000 7ff567fff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff55406ace0 0x7ff55406d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c00 tx=0x7ff558008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:54.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:54.784+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff55c0580d0 con 0x7ff5680b81c0 2023-07-06T21:40:55.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.027+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff5680d37c0 con 0x7ff5680b81c0 2023-07-06T21:40:55.140 INFO:tasks.ceph.mds.a.smithi084.stderr:2023-07-06T21:40:55.138+0000 7fbed37f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T21:40:55.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.960+0000 7ff564ff9700 1 -- 172.21.15.84:0/2613707395 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff55c057c60 con 0x7ff5680b81c0 2023-07-06T21:40:55.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff55406ace0 msgr2=0x7ff55406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff55406ace0 0x7ff55406d190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff558000c00 tx=0x7ff558008030 comp rx=0 tx=0).stop 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 msgr2=0x7ff5680cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 0x7ff5680cc100 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00e9b0 tx=0x7ff55c00ed70 comp rx=0 tx=0).stop 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 shutdown_connections 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff55406ace0 0x7ff55406d190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5680ceda0 0x7ff5681f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:55.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5680b1a80 0x7ff5680c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:55.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 --2- 172.21.15.84:0/2613707395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff5680b81c0 0x7ff5680cc100 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:55.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 >> 172.21.15.84:0/2613707395 conn(0x7ff56806b2b0 msgr2=0x7ff5680b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:55.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 shutdown_connections 2023-07-06T21:40:55.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:55.963+0000 7ff56e3ff700 1 -- 172.21.15.84:0/2613707395 wait complete. 2023-07-06T21:40:55.985 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T21:40:55.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T21:40:56.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.183+0000 7fd9021ff700 1 Processor -- start 2023-07-06T21:40:56.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.183+0000 7fd9021ff700 1 -- start start 2023-07-06T21:40:56.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.184+0000 7fd9021ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc0cf280 0x7fd8fc0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:56.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.184+0000 7fd9021ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0c94b0 con 0x7fd8fc069e90 2023-07-06T21:40:56.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.184+0000 7fd9021ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8fc0ca9b0 con 0x7fd8fc06dde0 2023-07-06T21:40:56.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.184+0000 7fd9021ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0cac90 con 0x7fd8fc0cf280 2023-07-06T21:40:56.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fb7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd8fc069e90 0x7fd8fc06ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48268/0 (socket says 172.21.15.84:48268) 2023-07-06T21:40:56.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fb7fe700 1 -- 172.21.15.84:0/2589419790 learned_addr learned my addr 172.21.15.84:0/2589419790 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:56.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fbfff700 1 --2- 172.21.15.84:0/2589419790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc0cf280 0x7fd8fc0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:56.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719731550 0 0) 0x7fd8fc0c94b0 con 0x7fd8fc069e90 2023-07-06T21:40:56.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e40035b0 con 0x7fd8fc069e90 2023-07-06T21:40:56.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235910028 0 0) 0x7fd8fc0ca9b0 con 0x7fd8fc06dde0 2023-07-06T21:40:56.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8fc0c94b0 con 0x7fd8fc06dde0 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671052079 0 0) 0x7fd8e40035b0 con 0x7fd8fc069e90 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fbfff700 1 -- 172.21.15.84:0/2589419790 >> v1:172.21.15.107:6789/0 conn(0x7fd8fc06dde0 legacy=0x7fd8fc06a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fbfff700 1 -- 172.21.15.84:0/2589419790 >> v1:172.21.15.84:6789/0 conn(0x7fd8fc069e90 legacy=0x7fd8fc06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.185+0000 7fd8fbfff700 1 -- 172.21.15.84:0/2589419790 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8fc0caf70 con 0x7fd8fc0cf280 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd8fbfff700 1 --2- 172.21.15.84:0/2589419790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc0cf280 0x7fd8fc0b1650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009dc0 tx=0x7fd8f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1160d3b61253a816 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f400e070 con 0x7fd8fc0cf280 2023-07-06T21:40:56.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f4003a20 con 0x7fd8fc0cf280 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd8fa7fc700 1 -- 172.21.15.84:0/2589419790 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec001670 con 0x7fd8fc0cf280 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd9021ff700 1 -- 172.21.15.84:0/2589419790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc0cf280 msgr2=0x7fd8fc0b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2589419790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc0cf280 0x7fd8fc0b1650 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4009dc0 tx=0x7fd8f400b330 comp rx=0 tx=0).stop 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd9021ff700 1 -- 172.21.15.84:0/2589419790 shutdown_connections 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2589419790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc0cf280 0x7fd8fc0b1650 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.186+0000 7fd9021ff700 1 -- 172.21.15.84:0/2589419790 >> 172.21.15.84:0/2589419790 conn(0x7fd8fc0cbf10 msgr2=0x7fd8fc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.187+0000 7fd9021ff700 1 -- 172.21.15.84:0/2589419790 shutdown_connections 2023-07-06T21:40:56.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.187+0000 7fd9021ff700 1 -- 172.21.15.84:0/2589419790 wait complete. 2023-07-06T21:40:56.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.188+0000 7fd9021ff700 1 Processor -- start 2023-07-06T21:40:56.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.188+0000 7fd9021ff700 1 -- start start 2023-07-06T21:40:56.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd9021ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 0x7fd8fc0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:56.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd9021ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc06dde0 0x7fd8fc152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:56.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd9021ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8fc0cf280 0x7fd8fc0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:56.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd9021ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc15b360 con 0x7fd8fc069e90 2023-07-06T21:40:56.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd9021ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc15be00 con 0x7fd8fc0cf280 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd9021ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8fc0adc50 con 0x7fd8fc06dde0 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.189+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 0x7fd8fc0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 0x7fd8fc0ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53190/0 (socket says 172.21.15.84:53190) 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 -- 172.21.15.84:0/2978700372 learned_addr learned my addr 172.21.15.84:0/2978700372 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8faffd700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc06dde0 0x7fd8fc152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fbfff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8fc0cf280 0x7fd8fc0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:56.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 -- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc06dde0 msgr2=0x7fd8fc152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc06dde0 0x7fd8fc152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 -- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8fc0cf280 msgr2=0x7fd8fc0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8fc0cf280 0x7fd8fc0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 -- 172.21.15.84:0/2978700372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f4009a70 con 0x7fd8fc069e90 2023-07-06T21:40:56.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8faffd700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc06dde0 0x7fd8fc152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:56.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fbfff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8fc0cf280 0x7fd8fc0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:56.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8fb7fe700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 0x7fd8fc0ad710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c6b0 tx=0x7fd8ec00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:56.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec004c80 con 0x7fd8fc069e90 2023-07-06T21:40:56.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ec004e00 con 0x7fd8fc069e90 2023-07-06T21:40:56.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ec007310 con 0x7fd8fc069e90 2023-07-06T21:40:56.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.190+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8fc15d5d0 con 0x7fd8fc069e90 2023-07-06T21:40:56.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.191+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8fc15b600 con 0x7fd8fc069e90 2023-07-06T21:40:56.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.192+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8fc04e1d0 con 0x7fd8fc069e90 2023-07-06T21:40:56.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.193+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd8ec007470 con 0x7fd8fc069e90 2023-07-06T21:40:56.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.194+0000 7fd8e3fff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd8e806ace0 0x7fd8e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.194+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd8ec0892a0 con 0x7fd8fc069e90 2023-07-06T21:40:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.194+0000 7fd8faffd700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd8e806ace0 0x7fd8e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.194+0000 7fd8faffd700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd8e806ace0 0x7fd8e806d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0007fd0 tx=0x7fd8f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.198+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8ec054340 con 0x7fd8fc069e90 2023-07-06T21:40:56.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.439+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd8fc0c8c30 con 0x7fd8fc069e90 2023-07-06T21:40:56.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.968+0000 7fd8e3fff700 1 -- 172.21.15.84:0/2978700372 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd8ec057960 con 0x7fd8fc069e90 2023-07-06T21:40:56.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd8e806ace0 msgr2=0x7fd8e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd8e806ace0 0x7fd8e806d190 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0007fd0 tx=0x7fd8f0009040 comp rx=0 tx=0).stop 2023-07-06T21:40:56.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 msgr2=0x7fd8fc0ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 0x7fd8fc0ad710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec00c6b0 tx=0x7fd8ec00ca70 comp rx=0 tx=0).stop 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 shutdown_connections 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd8e806ace0 0x7fd8e806d190 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8fc0cf280 0x7fd8fc0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8fc069e90 0x7fd8fc0ad710 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 --2- 172.21.15.84:0/2978700372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8fc06dde0 0x7fd8fc152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 >> 172.21.15.84:0/2978700372 conn(0x7fd8fc0cbf10 msgr2=0x7fd8fc070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:56.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 shutdown_connections 2023-07-06T21:40:56.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:56.971+0000 7fd9021ff700 1 -- 172.21.15.84:0/2978700372 wait complete. 2023-07-06T21:40:56.992 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T21:40:56.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:40:57.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.182+0000 7fe4e49ff700 1 Processor -- start 2023-07-06T21:40:57.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.182+0000 7fe4e49ff700 1 -- start start 2023-07-06T21:40:57.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.183+0000 7fe4e49ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe4e0159130 0x7fe4e00c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.183+0000 7fe4e49ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe4e0070940 0x7fe4e00c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4e49ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e015d930 con 0x7fe4e00c5000 2023-07-06T21:40:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4e49ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e015cd20 con 0x7fe4e0159130 2023-07-06T21:40:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4e49ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e015d000 con 0x7fe4e0070940 2023-07-06T21:40:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4de59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe4e0159130 0x7fe4e00c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4d7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe4e00c5000 0x7fe4e0070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48284/0 (socket says 172.21.15.84:48284) 2023-07-06T21:40:57.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4d7fff700 1 -- 172.21.15.84:0/1244991142 learned_addr learned my addr 172.21.15.84:0/1244991142 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:57.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4ded9d700 1 --2- 172.21.15.84:0/1244991142 >> v2:172.21.15.107:3300/0 conn(0x7fe4e0070940 0x7fe4e00c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4de59c700 1 -- 172.21.15.84:0/1244991142 >> v2:172.21.15.107:3300/0 conn(0x7fe4e0070940 msgr2=0x7fe4e00c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:57.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4de59c700 1 --2- 172.21.15.84:0/1244991142 >> v2:172.21.15.107:3300/0 conn(0x7fe4e0070940 0x7fe4e00c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4de59c700 1 -- 172.21.15.84:0/1244991142 >> v1:172.21.15.84:6789/0 conn(0x7fe4e00c5000 legacy=0x7fe4e0070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4de59c700 1 -- 172.21.15.84:0/1244991142 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e015d2e0 con 0x7fe4e0159130 2023-07-06T21:40:57.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4ded9d700 1 --2- 172.21.15.84:0/1244991142 >> v2:172.21.15.107:3300/0 conn(0x7fe4e0070940 0x7fe4e00c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:57.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.184+0000 7fe4ddd9b700 1 -- 172.21.15.84:0/1244991142 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106583515 0 0) 0x7fe4e015d930 con 0x7fe4e00c5000 2023-07-06T21:40:57.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4de59c700 1 --2- 172.21.15.84:0/1244991142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e0159130 0x7fe4e00c4ac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe4e006e470 tx=0x7fe4d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7b05f4199c467b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:57.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4ddd9b700 1 -- 172.21.15.84:0/1244991142 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0011830 con 0x7fe4e0159130 2023-07-06T21:40:57.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4ddd9b700 1 -- 172.21.15.84:0/1244991142 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d0011990 con 0x7fe4e0159130 2023-07-06T21:40:57.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4ddd9b700 1 -- 172.21.15.84:0/1244991142 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0011d20 con 0x7fe4e0159130 2023-07-06T21:40:57.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4e49ff700 1 -- 172.21.15.84:0/1244991142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e0159130 msgr2=0x7fe4e00c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/1244991142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e0159130 0x7fe4e00c4ac0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe4e006e470 tx=0x7fe4d0009b20 comp rx=0 tx=0).stop 2023-07-06T21:40:57.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4e49ff700 1 -- 172.21.15.84:0/1244991142 shutdown_connections 2023-07-06T21:40:57.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/1244991142 >> v2:172.21.15.107:3300/0 conn(0x7fe4e0070940 0x7fe4e00c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/1244991142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e0159130 0x7fe4e00c4ac0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.185+0000 7fe4e49ff700 1 -- 172.21.15.84:0/1244991142 >> 172.21.15.84:0/1244991142 conn(0x7fe4e00cf3d0 msgr2=0x7fe4e00cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:57.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.186+0000 7fe4e49ff700 1 -- 172.21.15.84:0/1244991142 shutdown_connections 2023-07-06T21:40:57.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.186+0000 7fe4e49ff700 1 -- 172.21.15.84:0/1244991142 wait complete. 2023-07-06T21:40:57.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.188+0000 7fe4e49ff700 1 Processor -- start 2023-07-06T21:40:57.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.188+0000 7fe4e49ff700 1 -- start start 2023-07-06T21:40:57.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.189+0000 7fe4e49ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4e0070940 0x7fe4e015b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.189+0000 7fe4e49ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 0x7fe4e00ac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.189+0000 7fe4e49ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0159130 0x7fe4e00acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.189+0000 7fe4e49ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e015a780 con 0x7fe4e00c5000 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.189+0000 7fe4e49ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e014eba0 con 0x7fe4e0159130 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.189+0000 7fe4e49ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e01f2e80 con 0x7fe4e0070940 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 0x7fe4e00ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 0x7fe4e00ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53202/0 (socket says 172.21.15.84:53202) 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 -- 172.21.15.84:0/3680908086 learned_addr learned my addr 172.21.15.84:0/3680908086 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:57.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4ded9d700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0159130 0x7fe4e00acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4de59c700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4e0070940 0x7fe4e015b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 -- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4e0070940 msgr2=0x7fe4e015b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4e0070940 0x7fe4e015b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 -- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0159130 msgr2=0x7fe4e00acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0159130 0x7fe4e00acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 -- 172.21.15.84:0/3680908086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00094a0 con 0x7fe4e00c5000 2023-07-06T21:40:57.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.190+0000 7fe4d7fff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 0x7fe4e00ac850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c5a0 tx=0x7fe4c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.191+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004c80 con 0x7fe4e00c5000 2023-07-06T21:40:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.191+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c8004de0 con 0x7fe4e00c5000 2023-07-06T21:40:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.191+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8007310 con 0x7fe4e00c5000 2023-07-06T21:40:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.191+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e00b1090 con 0x7fe4e00c5000 2023-07-06T21:40:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.191+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e015a990 con 0x7fe4e00c5000 2023-07-06T21:40:57.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.192+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e00ad820 con 0x7fe4e00c5000 2023-07-06T21:40:57.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.193+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe4c8007470 con 0x7fe4e00c5000 2023-07-06T21:40:57.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.194+0000 7fe4d6ffd700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe4c006ace0 0x7fe4c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.194+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4c8089180 con 0x7fe4e00c5000 2023-07-06T21:40:57.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.194+0000 7fe4de59c700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe4c006ace0 0x7fe4c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.194+0000 7fe4de59c700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe4c006ace0 0x7fe4c006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00090b0 tx=0x7fe4d0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:57.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.198+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe4c8054190 con 0x7fe4e00c5000 2023-07-06T21:40:57.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.439+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4e0071400 con 0x7fe4e00c5000 2023-07-06T21:40:57.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.439+0000 7fe4d6ffd700 1 -- 172.21.15.84:0/3680908086 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fe4c80577b0 con 0x7fe4e00c5000 2023-07-06T21:40:57.442 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:57.442 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:40:56.959728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:40:57.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe4c006ace0 msgr2=0x7fe4c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe4c006ace0 0x7fe4c006d190 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00090b0 tx=0x7fe4d0008030 comp rx=0 tx=0).stop 2023-07-06T21:40:57.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 msgr2=0x7fe4e00ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 0x7fe4e00ac850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800c5a0 tx=0x7fe4c800c8b0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 shutdown_connections 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe4c006ace0 0x7fe4c006d190 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e0159130 0x7fe4e00acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4e0070940 0x7fe4e015b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 --2- 172.21.15.84:0/3680908086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4e00c5000 0x7fe4e00ac850 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 >> 172.21.15.84:0/3680908086 conn(0x7fe4e00cf3d0 msgr2=0x7fe4e00ca470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:57.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 shutdown_connections 2023-07-06T21:40:57.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.442+0000 7fe4e49ff700 1 -- 172.21.15.84:0/3680908086 wait complete. 2023-07-06T21:40:57.446 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 15 2023-07-06T21:40:57.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:40:57.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.654+0000 7fce693ff700 1 Processor -- start 2023-07-06T21:40:57.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.654+0000 7fce693ff700 1 -- start start 2023-07-06T21:40:57.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.655+0000 7fce693ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fce640d3710 0x7fce640cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.655+0000 7fce693ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.655+0000 7fce693ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce640cd910 con 0x7fce640b83c0 2023-07-06T21:40:57.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.655+0000 7fce693ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce640cff10 con 0x7fce640d3710 2023-07-06T21:40:57.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.655+0000 7fce693ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce640d0a80 con 0x7fce640cfa90 2023-07-06T21:40:57.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce62ffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fce640b83c0 0x7fce640cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48294/0 (socket says 172.21.15.84:48294) 2023-07-06T21:40:57.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce62ffd700 1 -- 172.21.15.84:0/1185734613 learned_addr learned my addr 172.21.15.84:0/1185734613 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:57.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce627fc700 1 --2- 172.21.15.84:0/1185734613 >> v2:172.21.15.84:3300/0 conn(0x7fce640d3710 0x7fce640cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce637fe700 1 --2- 172.21.15.84:0/1185734613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce61ffb700 1 -- 172.21.15.84:0/1185734613 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948735476 0 0) 0x7fce54001110 con 0x7fce640b83c0 2023-07-06T21:40:57.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce61ffb700 1 -- 172.21.15.84:0/1185734613 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce4c0035b0 con 0x7fce640b83c0 2023-07-06T21:40:57.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce627fc700 1 -- 172.21.15.84:0/1185734613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 msgr2=0x7fce640d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:57.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce627fc700 1 --2- 172.21.15.84:0/1185734613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce627fc700 1 -- 172.21.15.84:0/1185734613 >> v1:172.21.15.84:6789/0 conn(0x7fce640b83c0 legacy=0x7fce640cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce627fc700 1 -- 172.21.15.84:0/1185734613 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640b8790 con 0x7fce640d3710 2023-07-06T21:40:57.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.656+0000 7fce627fc700 1 --2- 172.21.15.84:0/1185734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640d3710 0x7fce640cf550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fce58009840 tx=0x7fce58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136cba54c0c7b126 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:57.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce61ffb700 1 -- 172.21.15.84:0/1185734613 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58011830 con 0x7fce640d3710 2023-07-06T21:40:57.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce61ffb700 1 -- 172.21.15.84:0/1185734613 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58011990 con 0x7fce640d3710 2023-07-06T21:40:57.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce61ffb700 1 -- 172.21.15.84:0/1185734613 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58011d00 con 0x7fce640d3710 2023-07-06T21:40:57.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 -- 172.21.15.84:0/1185734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640d3710 msgr2=0x7fce640cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 --2- 172.21.15.84:0/1185734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640d3710 0x7fce640cf550 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fce58009840 tx=0x7fce58009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:57.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 -- 172.21.15.84:0/1185734613 shutdown_connections 2023-07-06T21:40:57.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 --2- 172.21.15.84:0/1185734613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 --2- 172.21.15.84:0/1185734613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640d3710 0x7fce640cf550 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 -- 172.21.15.84:0/1185734613 >> 172.21.15.84:0/1185734613 conn(0x7fce6406b350 msgr2=0x7fce6406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:57.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 -- 172.21.15.84:0/1185734613 shutdown_connections 2023-07-06T21:40:57.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.657+0000 7fce693ff700 1 -- 172.21.15.84:0/1185734613 wait complete. 2023-07-06T21:40:57.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.660+0000 7fce693ff700 1 Processor -- start 2023-07-06T21:40:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.660+0000 7fce693ff700 1 -- start start 2023-07-06T21:40:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce693ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 0x7fce6414da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce693ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce693ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce640d3710 0x7fce640b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce693ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6414df40 con 0x7fce640b83c0 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce693ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6415c930 con 0x7fce640d3710 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce693ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6415cb80 con 0x7fce640cfa90 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce62ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 0x7fce6414da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce62ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 0x7fce6414da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53224/0 (socket says 172.21.15.84:53224) 2023-07-06T21:40:57.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.661+0000 7fce62ffd700 1 -- 172.21.15.84:0/3779648759 learned_addr learned my addr 172.21.15.84:0/3779648759 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce627fc700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce637fe700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce640d3710 0x7fce640b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce62ffd700 1 -- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 msgr2=0x7fce640b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce62ffd700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce62ffd700 1 -- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce640d3710 msgr2=0x7fce640b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce62ffd700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce640d3710 0x7fce640b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce62ffd700 1 -- 172.21.15.84:0/3779648759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce580094a0 con 0x7fce640b83c0 2023-07-06T21:40:57.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce627fc700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce637fe700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce640d3710 0x7fce640b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce62ffd700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 0x7fce6414da00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c420 tx=0x7fce5400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54005d70 con 0x7fce640b83c0 2023-07-06T21:40:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540070d0 con 0x7fce640b83c0 2023-07-06T21:40:57.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6415ce60 con 0x7fce640b83c0 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54007480 con 0x7fce640b83c0 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.662+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6415d420 con 0x7fce640b83c0 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.663+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6415b290 con 0x7fce640b83c0 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.664+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fce540075e0 con 0x7fce640b83c0 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.666+0000 7fce4b7fe700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fce5006ace0 0x7fce5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.666+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fce54088890 con 0x7fce640b83c0 2023-07-06T21:40:57.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.666+0000 7fce627fc700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fce5006ace0 0x7fce5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.667+0000 7fce627fc700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fce5006ace0 0x7fce5006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce640d3b20 tx=0x7fce58008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:57.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.670+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fce540538a0 con 0x7fce640b83c0 2023-07-06T21:40:57.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.869+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce640cd910 con 0x7fce640b83c0 2023-07-06T21:40:57.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.870+0000 7fce4b7fe700 1 -- 172.21.15.84:0/3779648759 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18351 (secure 0 0 0) 0x7fce640cd910 con 0x7fce640b83c0 2023-07-06T21:40:57.872 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:57.873 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":37,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:40:54.659539+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:40:57.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fce5006ace0 msgr2=0x7fce5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fce5006ace0 0x7fce5006d190 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fce640d3b20 tx=0x7fce58008030 comp rx=0 tx=0).stop 2023-07-06T21:40:57.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 msgr2=0x7fce6414da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 0x7fce6414da00 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c420 tx=0x7fce5400c7e0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 shutdown_connections 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fce5006ace0 0x7fce5006d190 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce640d3710 0x7fce640b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fce640b83c0 0x7fce6414da00 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 --2- 172.21.15.84:0/3779648759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce640cfa90 0x7fce640b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:57.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 >> 172.21.15.84:0/3779648759 conn(0x7fce6406b350 msgr2=0x7fce640b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:57.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 shutdown_connections 2023-07-06T21:40:57.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:57.872+0000 7fce693ff700 1 -- 172.21.15.84:0/3779648759 wait complete. 2023-07-06T21:40:57.893 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T21:40:57.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:40:58.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.087+0000 7f242c3ff700 1 Processor -- start 2023-07-06T21:40:58.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.087+0000 7f242c3ff700 1 -- start start 2023-07-06T21:40:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.088+0000 7f242c3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f24240b83c0 0x7f24240cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.088+0000 7f242c3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24240d0a80 con 0x7f24240d3710 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.088+0000 7f242c3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24240b8790 con 0x7f24240cfc60 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.088+0000 7f242c3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24240b4e70 con 0x7f24240b83c0 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.088+0000 7f242999a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f24240d3710 0x7f24240cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48304/0 (socket says 172.21.15.84:48304) 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.088+0000 7f242999a700 1 -- 172.21.15.84:0/4246784433 learned_addr learned my addr 172.21.15.84:0/4246784433 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408021137 0 0) 0x7f24240d0a80 con 0x7f24240d3710 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24100035b0 con 0x7f24240d3710 2023-07-06T21:40:58.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f242a19b700 1 --2- 172.21.15.84:0/4246784433 >> v2:172.21.15.107:3300/0 conn(0x7f24240b83c0 0x7f24240cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3566652239 0 0) 0x7f24100035b0 con 0x7f24240d3710 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 >> v2:172.21.15.107:3300/0 conn(0x7f24240b83c0 msgr2=0x7f24240cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 --2- 172.21.15.84:0/4246784433 >> v2:172.21.15.107:3300/0 conn(0x7f24240b83c0 0x7f24240cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 >> v1:172.21.15.107:6789/0 conn(0x7f24240cfc60 legacy=0x7f24240d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24240b5150 con 0x7f24240d3710 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f2414001670 con 0x7f24240d3710 2023-07-06T21:40:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 >> v1:172.21.15.84:6789/0 conn(0x7f24240d3710 legacy=0x7f24240cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2410003a00 0x7f2410003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2410004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2410005200 0x7f241000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240b4e70 con 0x7f2410005200 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241000c0e0 con 0x7f2410004850 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.089+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241000c2a0 con 0x7f2410003a00 2023-07-06T21:40:58.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242a99c700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2410005200 0x7f241000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2410004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242a19b700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2410003a00 0x7f2410003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 -- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2410003a00 msgr2=0x7f2410003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2410003a00 0x7f2410003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 -- 172.21.15.84:0/4246784433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2410005200 msgr2=0x7f241000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2410005200 0x7f241000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 -- 172.21.15.84:0/4246784433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24100035b0 con 0x7f2410004850 2023-07-06T21:40:58.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242a99c700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2410005200 0x7f241000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242a99c700 1 -- 172.21.15.84:0/4246784433 reap_dead start 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.090+0000 7f242999a700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2410004cc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f24240d3b20 tx=0x7f2410005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a435051c98f3c55d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414007b70 con 0x7f2410004850 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400cd50 con 0x7f2410004850 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f2429199700 1 -- 172.21.15.84:0/4246784433 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414005ba0 con 0x7f2410004850 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 -- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 msgr2=0x7f2410004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2410004cc0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f24240d3b20 tx=0x7f2410005880 comp rx=0 tx=0).stop 2023-07-06T21:40:58.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 -- 172.21.15.84:0/4246784433 shutdown_connections 2023-07-06T21:40:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 --2- 172.21.15.84:0/4246784433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2410004cc0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 -- 172.21.15.84:0/4246784433 >> 172.21.15.84:0/4246784433 conn(0x7f242406b350 msgr2=0x7f242406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 -- 172.21.15.84:0/4246784433 shutdown_connections 2023-07-06T21:40:58.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.091+0000 7f242c3ff700 1 -- 172.21.15.84:0/4246784433 wait complete. 2023-07-06T21:40:58.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.093+0000 7f242c3ff700 1 Processor -- start 2023-07-06T21:40:58.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.093+0000 7f242c3ff700 1 -- start start 2023-07-06T21:40:58.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242c3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2424155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242c3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 0x7f24240b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242c3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2424154830 0x7f24240b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242c3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424155d70 con 0x7f24241f5ef0 2023-07-06T21:40:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242c3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424155a90 con 0x7f2410004850 2023-07-06T21:40:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242c3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24241f5410 con 0x7f2424154830 2023-07-06T21:40:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242999a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 0x7f24240b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242999a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 0x7f24240b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53234/0 (socket says 172.21.15.84:53234) 2023-07-06T21:40:58.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.094+0000 7f242999a700 1 -- 172.21.15.84:0/3174320052 learned_addr learned my addr 172.21.15.84:0/3174320052 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:58.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242a19b700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2424155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242a99c700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2424154830 0x7f24240b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242999a700 1 -- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2424154830 msgr2=0x7f24240b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242999a700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2424154830 0x7f24240b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242999a700 1 -- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 msgr2=0x7f2424155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242999a700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2424155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242999a700 1 -- 172.21.15.84:0/3174320052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24241f56f0 con 0x7f24241f5ef0 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242999a700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 0x7f24240b69c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f241000c270 tx=0x7f2414001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414001370 con 0x7f24241f5ef0 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24140014d0 con 0x7f24241f5ef0 2023-07-06T21:40:58.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24241f59d0 con 0x7f24241f5ef0 2023-07-06T21:40:58.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.095+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2414007310 con 0x7f24241f5ef0 2023-07-06T21:40:58.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.096+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242415c770 con 0x7f24241f5ef0 2023-07-06T21:40:58.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.096+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24241f5b90 con 0x7f24241f5ef0 2023-07-06T21:40:58.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.098+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2414007470 con 0x7f24241f5ef0 2023-07-06T21:40:58.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.098+0000 7f241affd700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f240806ace0 0x7f240806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.099+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f241408a980 con 0x7f24241f5ef0 2023-07-06T21:40:58.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.099+0000 7f242a19b700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f240806ace0 0x7f240806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.099+0000 7f242a19b700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f240806ace0 0x7f240806d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2424159ef0 tx=0x7f2420008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:58.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.103+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2414055ac0 con 0x7f24241f5ef0 2023-07-06T21:40:58.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.295+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24241f5290 con 0x7f240806ace0 2023-07-06T21:40:58.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.305+0000 7f241affd700 1 -- 172.21.15.84:0/3174320052 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311157 (secure 0 0 0) 0x7f2420009890 con 0x7f240806ace0 2023-07-06T21:40:58.311 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:58.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f240806ace0 msgr2=0x7f240806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f240806ace0 0x7f240806d190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2424159ef0 tx=0x7f2420008040 comp rx=0 tx=0).stop 2023-07-06T21:40:58.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 msgr2=0x7f24240b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 0x7f24240b69c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f241000c270 tx=0x7f2414001930 comp rx=0 tx=0).stop 2023-07-06T21:40:58.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 shutdown_connections 2023-07-06T21:40:58.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f240806ace0 0x7f240806d190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2424154830 0x7f24240b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2410004850 0x7f2424155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 --2- 172.21.15.84:0/3174320052 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f24241f5ef0 0x7f24240b69c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 >> 172.21.15.84:0/3174320052 conn(0x7f242406b350 msgr2=0x7f242415fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:58.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 shutdown_connections 2023-07-06T21:40:58.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.310+0000 7f242c3ff700 1 -- 172.21.15.84:0/3174320052 wait complete. 2023-07-06T21:40:58.315 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:40:58.331 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-07-06T21:40:58.154293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220132,"kb_used_data":4516,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754588,"statfs":{"total":773094113280,"available":772868698112,"internally_reserved":0,"allocated":4624384,"data_stored":2587545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991414"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684248+0000","last_change":"2023-07-06T21:40:53.685275+0000","last_active":"2023-07-06T21:40:55.684248+0000","last_peered":"2023-07-06T21:40:55.684248+0000","last_clean":"2023-07-06T21:40:55.684248+0000","last_became_active":"2023-07-06T21:40:53.685007+0000","last_became_peered":"2023-07-06T21:40:53.685007+0000","last_unstale":"2023-07-06T21:40:55.684248+0000","last_undegraded":"2023-07-06T21:40:55.684248+0000","last_fullsized":"2023-07-06T21:40:55.684248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:17:53.745382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684199+0000","last_change":"2023-07-06T21:40:53.688448+0000","last_active":"2023-07-06T21:40:55.684199+0000","last_peered":"2023-07-06T21:40:55.684199+0000","last_clean":"2023-07-06T21:40:55.684199+0000","last_became_active":"2023-07-06T21:40:53.688150+0000","last_became_peered":"2023-07-06T21:40:53.688150+0000","last_unstale":"2023-07-06T21:40:55.684199+0000","last_undegraded":"2023-07-06T21:40:55.684199+0000","last_fullsized":"2023-07-06T21:40:55.684199+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:39:14.937173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482163+0000","last_change":"2023-07-06T21:40:53.691316+0000","last_active":"2023-07-06T21:40:56.482163+0000","last_peered":"2023-07-06T21:40:56.482163+0000","last_clean":"2023-07-06T21:40:56.482163+0000","last_became_active":"2023-07-06T21:40:53.691051+0000","last_became_peered":"2023-07-06T21:40:53.691051+0000","last_unstale":"2023-07-06T21:40:56.482163+0000","last_undegraded":"2023-07-06T21:40:56.482163+0000","last_fullsized":"2023-07-06T21:40:56.482163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:04.716858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664295+0000","last_change":"2023-07-06T21:40:53.672857+0000","last_active":"2023-07-06T21:40:54.664295+0000","last_peered":"2023-07-06T21:40:54.664295+0000","last_clean":"2023-07-06T21:40:54.664295+0000","last_became_active":"2023-07-06T21:40:53.672557+0000","last_became_peered":"2023-07-06T21:40:53.672557+0000","last_unstale":"2023-07-06T21:40:54.664295+0000","last_undegraded":"2023-07-06T21:40:54.664295+0000","last_fullsized":"2023-07-06T21:40:54.664295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:41:46.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664230+0000","last_change":"2023-07-06T21:40:53.671963+0000","last_active":"2023-07-06T21:40:54.664230+0000","last_peered":"2023-07-06T21:40:54.664230+0000","last_clean":"2023-07-06T21:40:54.664230+0000","last_became_active":"2023-07-06T21:40:53.671695+0000","last_became_peered":"2023-07-06T21:40:53.671695+0000","last_unstale":"2023-07-06T21:40:54.664230+0000","last_undegraded":"2023-07-06T21:40:54.664230+0000","last_fullsized":"2023-07-06T21:40:54.664230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:21:02.102040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381518+0000","last_change":"2023-07-06T21:40:53.678253+0000","last_active":"2023-07-06T21:40:55.381518+0000","last_peered":"2023-07-06T21:40:55.381518+0000","last_clean":"2023-07-06T21:40:55.381518+0000","last_became_active":"2023-07-06T21:40:53.677998+0000","last_became_peered":"2023-07-06T21:40:53.677998+0000","last_unstale":"2023-07-06T21:40:55.381518+0000","last_undegraded":"2023-07-06T21:40:55.381518+0000","last_fullsized":"2023-07-06T21:40:55.381518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:07.534353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668015+0000","last_change":"2023-07-06T21:40:53.691770+0000","last_active":"2023-07-06T21:40:54.668015+0000","last_peered":"2023-07-06T21:40:54.668015+0000","last_clean":"2023-07-06T21:40:54.668015+0000","last_became_active":"2023-07-06T21:40:53.689786+0000","last_became_peered":"2023-07-06T21:40:53.689786+0000","last_unstale":"2023-07-06T21:40:54.668015+0000","last_undegraded":"2023-07-06T21:40:54.668015+0000","last_fullsized":"2023-07-06T21:40:54.668015+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:34:42.229390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.683876+0000","last_change":"2023-07-06T21:40:53.681428+0000","last_active":"2023-07-06T21:40:55.683876+0000","last_peered":"2023-07-06T21:40:55.683876+0000","last_clean":"2023-07-06T21:40:55.683876+0000","last_became_active":"2023-07-06T21:40:53.681141+0000","last_became_peered":"2023-07-06T21:40:53.681141+0000","last_unstale":"2023-07-06T21:40:55.683876+0000","last_undegraded":"2023-07-06T21:40:55.683876+0000","last_fullsized":"2023-07-06T21:40:55.683876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:34:05.003871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482299+0000","last_change":"2023-07-06T21:40:53.689527+0000","last_active":"2023-07-06T21:40:56.482299+0000","last_peered":"2023-07-06T21:40:56.482299+0000","last_clean":"2023-07-06T21:40:56.482299+0000","last_became_active":"2023-07-06T21:40:53.689216+0000","last_became_peered":"2023-07-06T21:40:53.689216+0000","last_unstale":"2023-07-06T21:40:56.482299+0000","last_undegraded":"2023-07-06T21:40:56.482299+0000","last_fullsized":"2023-07-06T21:40:56.482299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:54.151894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668381+0000","last_change":"2023-07-06T21:40:53.692783+0000","last_active":"2023-07-06T21:40:54.668381+0000","last_peered":"2023-07-06T21:40:54.668381+0000","last_clean":"2023-07-06T21:40:54.668381+0000","last_became_active":"2023-07-06T21:40:53.692496+0000","last_became_peered":"2023-07-06T21:40:53.692496+0000","last_unstale":"2023-07-06T21:40:54.668381+0000","last_undegraded":"2023-07-06T21:40:54.668381+0000","last_fullsized":"2023-07-06T21:40:54.668381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:37.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668277+0000","last_change":"2023-07-06T21:40:53.688460+0000","last_active":"2023-07-06T21:40:54.668277+0000","last_peered":"2023-07-06T21:40:54.668277+0000","last_clean":"2023-07-06T21:40:54.668277+0000","last_became_active":"2023-07-06T21:40:53.688195+0000","last_became_peered":"2023-07-06T21:40:53.688195+0000","last_unstale":"2023-07-06T21:40:54.668277+0000","last_undegraded":"2023-07-06T21:40:54.668277+0000","last_fullsized":"2023-07-06T21:40:54.668277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:22.655067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482066+0000","last_change":"2023-07-06T21:40:53.682035+0000","last_active":"2023-07-06T21:40:56.482066+0000","last_peered":"2023-07-06T21:40:56.482066+0000","last_clean":"2023-07-06T21:40:56.482066+0000","last_became_active":"2023-07-06T21:40:53.681671+0000","last_became_peered":"2023-07-06T21:40:53.681671+0000","last_unstale":"2023-07-06T21:40:56.482066+0000","last_undegraded":"2023-07-06T21:40:56.482066+0000","last_fullsized":"2023-07-06T21:40:56.482066+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:33.585902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668119+0000","last_change":"2023-07-06T21:40:53.692570+0000","last_active":"2023-07-06T21:40:54.668119+0000","last_peered":"2023-07-06T21:40:54.668119+0000","last_clean":"2023-07-06T21:40:54.668119+0000","last_became_active":"2023-07-06T21:40:53.692276+0000","last_became_peered":"2023-07-06T21:40:53.692276+0000","last_unstale":"2023-07-06T21:40:54.668119+0000","last_undegraded":"2023-07-06T21:40:54.668119+0000","last_fullsized":"2023-07-06T21:40:54.668119+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:03:53.052919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483096+0000","last_change":"2023-07-06T21:40:53.688112+0000","last_active":"2023-07-06T21:40:56.483096+0000","last_peered":"2023-07-06T21:40:56.483096+0000","last_clean":"2023-07-06T21:40:56.483096+0000","last_became_active":"2023-07-06T21:40:53.687820+0000","last_became_peered":"2023-07-06T21:40:53.687820+0000","last_unstale":"2023-07-06T21:40:56.483096+0000","last_undegraded":"2023-07-06T21:40:56.483096+0000","last_fullsized":"2023-07-06T21:40:56.483096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:28.801628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667836+0000","last_change":"2023-07-06T21:40:53.682885+0000","last_active":"2023-07-06T21:40:54.667836+0000","last_peered":"2023-07-06T21:40:54.667836+0000","last_clean":"2023-07-06T21:40:54.667836+0000","last_became_active":"2023-07-06T21:40:53.682538+0000","last_became_peered":"2023-07-06T21:40:53.682538+0000","last_unstale":"2023-07-06T21:40:54.667836+0000","last_undegraded":"2023-07-06T21:40:54.667836+0000","last_fullsized":"2023-07-06T21:40:54.667836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:31.464007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036552+0000","last_change":"2023-07-06T21:40:53.680201+0000","last_active":"2023-07-06T21:40:56.036552+0000","last_peered":"2023-07-06T21:40:56.036552+0000","last_clean":"2023-07-06T21:40:56.036552+0000","last_became_active":"2023-07-06T21:40:53.679930+0000","last_became_peered":"2023-07-06T21:40:53.679930+0000","last_unstale":"2023-07-06T21:40:56.036552+0000","last_undegraded":"2023-07-06T21:40:56.036552+0000","last_fullsized":"2023-07-06T21:40:56.036552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:01:33.946210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036773+0000","last_change":"2023-07-06T21:40:53.693208+0000","last_active":"2023-07-06T21:40:56.036773+0000","last_peered":"2023-07-06T21:40:56.036773+0000","last_clean":"2023-07-06T21:40:56.036773+0000","last_became_active":"2023-07-06T21:40:53.692337+0000","last_became_peered":"2023-07-06T21:40:53.692337+0000","last_unstale":"2023-07-06T21:40:56.036773+0000","last_undegraded":"2023-07-06T21:40:56.036773+0000","last_fullsized":"2023-07-06T21:40:56.036773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:35:46.789829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668442+0000","last_change":"2023-07-06T21:40:53.685670+0000","last_active":"2023-07-06T21:40:54.668442+0000","last_peered":"2023-07-06T21:40:54.668442+0000","last_clean":"2023-07-06T21:40:54.668442+0000","last_became_active":"2023-07-06T21:40:53.685349+0000","last_became_peered":"2023-07-06T21:40:53.685349+0000","last_unstale":"2023-07-06T21:40:54.668442+0000","last_undegraded":"2023-07-06T21:40:54.668442+0000","last_fullsized":"2023-07-06T21:40:54.668442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:13:27.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664189+0000","last_change":"2023-07-06T21:40:53.683998+0000","last_active":"2023-07-06T21:40:54.664189+0000","last_peered":"2023-07-06T21:40:54.664189+0000","last_clean":"2023-07-06T21:40:54.664189+0000","last_became_active":"2023-07-06T21:40:53.683736+0000","last_became_peered":"2023-07-06T21:40:53.683736+0000","last_unstale":"2023-07-06T21:40:54.664189+0000","last_undegraded":"2023-07-06T21:40:54.664189+0000","last_fullsized":"2023-07-06T21:40:54.664189+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:27:17.481351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667941+0000","last_change":"2023-07-06T21:40:53.687772+0000","last_active":"2023-07-06T21:40:54.667941+0000","last_peered":"2023-07-06T21:40:54.667941+0000","last_clean":"2023-07-06T21:40:54.667941+0000","last_became_active":"2023-07-06T21:40:53.687283+0000","last_became_peered":"2023-07-06T21:40:53.687283+0000","last_unstale":"2023-07-06T21:40:54.667941+0000","last_undegraded":"2023-07-06T21:40:54.667941+0000","last_fullsized":"2023-07-06T21:40:54.667941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:45:50.000441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668298+0000","last_change":"2023-07-06T21:40:53.682118+0000","last_active":"2023-07-06T21:40:54.668298+0000","last_peered":"2023-07-06T21:40:54.668298+0000","last_clean":"2023-07-06T21:40:54.668298+0000","last_became_active":"2023-07-06T21:40:53.681788+0000","last_became_peered":"2023-07-06T21:40:53.681788+0000","last_unstale":"2023-07-06T21:40:54.668298+0000","last_undegraded":"2023-07-06T21:40:54.668298+0000","last_fullsized":"2023-07-06T21:40:54.668298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:11:58.346462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482642+0000","last_change":"2023-07-06T21:40:53.681341+0000","last_active":"2023-07-06T21:40:56.482642+0000","last_peered":"2023-07-06T21:40:56.482642+0000","last_clean":"2023-07-06T21:40:56.482642+0000","last_became_active":"2023-07-06T21:40:53.680886+0000","last_became_peered":"2023-07-06T21:40:53.680886+0000","last_unstale":"2023-07-06T21:40:56.482642+0000","last_undegraded":"2023-07-06T21:40:56.482642+0000","last_fullsized":"2023-07-06T21:40:56.482642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:37.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036481+0000","last_change":"2023-07-06T21:40:53.688853+0000","last_active":"2023-07-06T21:40:56.036481+0000","last_peered":"2023-07-06T21:40:56.036481+0000","last_clean":"2023-07-06T21:40:56.036481+0000","last_became_active":"2023-07-06T21:40:53.688486+0000","last_became_peered":"2023-07-06T21:40:53.688486+0000","last_unstale":"2023-07-06T21:40:56.036481+0000","last_undegraded":"2023-07-06T21:40:56.036481+0000","last_fullsized":"2023-07-06T21:40:56.036481+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:09:05.954082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663893+0000","last_change":"2023-07-06T21:40:53.676302+0000","last_active":"2023-07-06T21:40:54.663893+0000","last_peered":"2023-07-06T21:40:54.663893+0000","last_clean":"2023-07-06T21:40:54.663893+0000","last_became_active":"2023-07-06T21:40:53.676007+0000","last_became_peered":"2023-07-06T21:40:53.676007+0000","last_unstale":"2023-07-06T21:40:54.663893+0000","last_undegraded":"2023-07-06T21:40:54.663893+0000","last_fullsized":"2023-07-06T21:40:54.663893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:44:00.532865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668558+0000","last_change":"2023-07-06T21:40:53.685340+0000","last_active":"2023-07-06T21:40:54.668558+0000","last_peered":"2023-07-06T21:40:54.668558+0000","last_clean":"2023-07-06T21:40:54.668558+0000","last_became_active":"2023-07-06T21:40:53.685016+0000","last_became_peered":"2023-07-06T21:40:53.685016+0000","last_unstale":"2023-07-06T21:40:54.668558+0000","last_undegraded":"2023-07-06T21:40:54.668558+0000","last_fullsized":"2023-07-06T21:40:54.668558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:06:42.264796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381914+0000","last_change":"2023-07-06T21:40:53.692733+0000","last_active":"2023-07-06T21:40:55.381914+0000","last_peered":"2023-07-06T21:40:55.381914+0000","last_clean":"2023-07-06T21:40:55.381914+0000","last_became_active":"2023-07-06T21:40:53.692433+0000","last_became_peered":"2023-07-06T21:40:53.692433+0000","last_unstale":"2023-07-06T21:40:55.381914+0000","last_undegraded":"2023-07-06T21:40:55.381914+0000","last_fullsized":"2023-07-06T21:40:55.381914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:11.828608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667780+0000","last_change":"2023-07-06T21:40:53.684666+0000","last_active":"2023-07-06T21:40:54.667780+0000","last_peered":"2023-07-06T21:40:54.667780+0000","last_clean":"2023-07-06T21:40:54.667780+0000","last_became_active":"2023-07-06T21:40:53.684371+0000","last_became_peered":"2023-07-06T21:40:53.684371+0000","last_unstale":"2023-07-06T21:40:54.667780+0000","last_undegraded":"2023-07-06T21:40:54.667780+0000","last_fullsized":"2023-07-06T21:40:54.667780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:31:41.695094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668337+0000","last_change":"2023-07-06T21:40:53.689521+0000","last_active":"2023-07-06T21:40:54.668337+0000","last_peered":"2023-07-06T21:40:54.668337+0000","last_clean":"2023-07-06T21:40:54.668337+0000","last_became_active":"2023-07-06T21:40:53.688866+0000","last_became_peered":"2023-07-06T21:40:53.688866+0000","last_unstale":"2023-07-06T21:40:54.668337+0000","last_undegraded":"2023-07-06T21:40:54.668337+0000","last_fullsized":"2023-07-06T21:40:54.668337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:24.396950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668290+0000","last_change":"2023-07-06T21:40:53.685506+0000","last_active":"2023-07-06T21:40:54.668290+0000","last_peered":"2023-07-06T21:40:54.668290+0000","last_clean":"2023-07-06T21:40:54.668290+0000","last_became_active":"2023-07-06T21:40:53.685138+0000","last_became_peered":"2023-07-06T21:40:53.685138+0000","last_unstale":"2023-07-06T21:40:54.668290+0000","last_undegraded":"2023-07-06T21:40:54.668290+0000","last_fullsized":"2023-07-06T21:40:54.668290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:41:46.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667807+0000","last_change":"2023-07-06T21:40:53.684696+0000","last_active":"2023-07-06T21:40:54.667807+0000","last_peered":"2023-07-06T21:40:54.667807+0000","last_clean":"2023-07-06T21:40:54.667807+0000","last_became_active":"2023-07-06T21:40:53.684366+0000","last_became_peered":"2023-07-06T21:40:53.684366+0000","last_unstale":"2023-07-06T21:40:54.667807+0000","last_undegraded":"2023-07-06T21:40:54.667807+0000","last_fullsized":"2023-07-06T21:40:54.667807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:06:26.004386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036363+0000","last_change":"2023-07-06T21:40:53.678093+0000","last_active":"2023-07-06T21:40:56.036363+0000","last_peered":"2023-07-06T21:40:56.036363+0000","last_clean":"2023-07-06T21:40:56.036363+0000","last_became_active":"2023-07-06T21:40:53.677832+0000","last_became_peered":"2023-07-06T21:40:53.677832+0000","last_unstale":"2023-07-06T21:40:56.036363+0000","last_undegraded":"2023-07-06T21:40:56.036363+0000","last_fullsized":"2023-07-06T21:40:56.036363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:16:43.933175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381523+0000","last_change":"2023-07-06T21:40:53.681621+0000","last_active":"2023-07-06T21:40:55.381523+0000","last_peered":"2023-07-06T21:40:55.381523+0000","last_clean":"2023-07-06T21:40:55.381523+0000","last_became_active":"2023-07-06T21:40:53.681341+0000","last_became_peered":"2023-07-06T21:40:53.681341+0000","last_unstale":"2023-07-06T21:40:55.381523+0000","last_undegraded":"2023-07-06T21:40:55.381523+0000","last_fullsized":"2023-07-06T21:40:55.381523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:52:48.741999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036888+0000","last_change":"2023-07-06T21:40:53.692883+0000","last_active":"2023-07-06T21:40:56.036888+0000","last_peered":"2023-07-06T21:40:56.036888+0000","last_clean":"2023-07-06T21:40:56.036888+0000","last_became_active":"2023-07-06T21:40:53.692585+0000","last_became_peered":"2023-07-06T21:40:53.692585+0000","last_unstale":"2023-07-06T21:40:56.036888+0000","last_undegraded":"2023-07-06T21:40:56.036888+0000","last_fullsized":"2023-07-06T21:40:56.036888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:11:58.346462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668531+0000","last_change":"2023-07-06T21:40:53.185932+0000","last_active":"2023-07-06T21:40:54.668531+0000","last_peered":"2023-07-06T21:40:54.668531+0000","last_clean":"2023-07-06T21:40:54.668531+0000","last_became_active":"2023-07-06T21:40:53.185631+0000","last_became_peered":"2023-07-06T21:40:53.185631+0000","last_unstale":"2023-07-06T21:40:54.668531+0000","last_undegraded":"2023-07-06T21:40:54.668531+0000","last_fullsized":"2023-07-06T21:40:54.668531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:50:41.968014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381874+0000","last_change":"2023-07-06T21:40:53.693028+0000","last_active":"2023-07-06T21:40:55.381874+0000","last_peered":"2023-07-06T21:40:55.381874+0000","last_clean":"2023-07-06T21:40:55.381874+0000","last_became_active":"2023-07-06T21:40:53.692667+0000","last_became_peered":"2023-07-06T21:40:53.692667+0000","last_unstale":"2023-07-06T21:40:55.381874+0000","last_undegraded":"2023-07-06T21:40:55.381874+0000","last_fullsized":"2023-07-06T21:40:55.381874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:01:33.946210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482185+0000","last_change":"2023-07-06T21:40:53.190299+0000","last_active":"2023-07-06T21:40:56.482185+0000","last_peered":"2023-07-06T21:40:56.482185+0000","last_clean":"2023-07-06T21:40:56.482185+0000","last_became_active":"2023-07-06T21:40:53.189951+0000","last_became_peered":"2023-07-06T21:40:53.189951+0000","last_unstale":"2023-07-06T21:40:56.482185+0000","last_undegraded":"2023-07-06T21:40:56.482185+0000","last_fullsized":"2023-07-06T21:40:56.482185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:09:14.716482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483056+0000","last_change":"2023-07-06T21:40:53.687852+0000","last_active":"2023-07-06T21:40:56.483056+0000","last_peered":"2023-07-06T21:40:56.483056+0000","last_clean":"2023-07-06T21:40:56.483056+0000","last_became_active":"2023-07-06T21:40:53.687513+0000","last_became_peered":"2023-07-06T21:40:53.687513+0000","last_unstale":"2023-07-06T21:40:56.483056+0000","last_undegraded":"2023-07-06T21:40:56.483056+0000","last_fullsized":"2023-07-06T21:40:56.483056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:52.030881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381892+0000","last_change":"2023-07-06T21:40:53.183298+0000","last_active":"2023-07-06T21:40:55.381892+0000","last_peered":"2023-07-06T21:40:55.381892+0000","last_clean":"2023-07-06T21:40:55.381892+0000","last_became_active":"2023-07-06T21:40:53.183047+0000","last_became_peered":"2023-07-06T21:40:53.183047+0000","last_unstale":"2023-07-06T21:40:55.381892+0000","last_undegraded":"2023-07-06T21:40:55.381892+0000","last_fullsized":"2023-07-06T21:40:55.381892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:16:42.925087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684291+0000","last_change":"2023-07-06T21:40:53.687980+0000","last_active":"2023-07-06T21:40:55.684291+0000","last_peered":"2023-07-06T21:40:55.684291+0000","last_clean":"2023-07-06T21:40:55.684291+0000","last_became_active":"2023-07-06T21:40:53.687481+0000","last_became_peered":"2023-07-06T21:40:53.687481+0000","last_unstale":"2023-07-06T21:40:55.684291+0000","last_undegraded":"2023-07-06T21:40:55.684291+0000","last_fullsized":"2023-07-06T21:40:55.684291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:16:43.933175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482902+0000","last_change":"2023-07-06T21:40:53.187761+0000","last_active":"2023-07-06T21:40:56.482902+0000","last_peered":"2023-07-06T21:40:56.482902+0000","last_clean":"2023-07-06T21:40:56.482902+0000","last_became_active":"2023-07-06T21:40:53.187506+0000","last_became_peered":"2023-07-06T21:40:53.187506+0000","last_unstale":"2023-07-06T21:40:56.482902+0000","last_undegraded":"2023-07-06T21:40:56.482902+0000","last_fullsized":"2023-07-06T21:40:56.482902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:02:52.869881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.481828+0000","last_change":"2023-07-06T21:40:53.683553+0000","last_active":"2023-07-06T21:40:56.481828+0000","last_peered":"2023-07-06T21:40:56.481828+0000","last_clean":"2023-07-06T21:40:56.481828+0000","last_became_active":"2023-07-06T21:40:53.683289+0000","last_became_peered":"2023-07-06T21:40:53.683289+0000","last_unstale":"2023-07-06T21:40:56.481828+0000","last_undegraded":"2023-07-06T21:40:56.481828+0000","last_fullsized":"2023-07-06T21:40:56.481828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:18.670130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668232+0000","last_change":"2023-07-06T21:40:53.185220+0000","last_active":"2023-07-06T21:40:54.668232+0000","last_peered":"2023-07-06T21:40:54.668232+0000","last_clean":"2023-07-06T21:40:54.668232+0000","last_became_active":"2023-07-06T21:40:53.184950+0000","last_became_peered":"2023-07-06T21:40:53.184950+0000","last_unstale":"2023-07-06T21:40:54.668232+0000","last_undegraded":"2023-07-06T21:40:54.668232+0000","last_fullsized":"2023-07-06T21:40:54.668232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:10:57.566657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668562+0000","last_change":"2023-07-06T21:40:53.684111+0000","last_active":"2023-07-06T21:40:54.668562+0000","last_peered":"2023-07-06T21:40:54.668562+0000","last_clean":"2023-07-06T21:40:54.668562+0000","last_became_active":"2023-07-06T21:40:53.683833+0000","last_became_peered":"2023-07-06T21:40:53.683833+0000","last_unstale":"2023-07-06T21:40:54.668562+0000","last_undegraded":"2023-07-06T21:40:54.668562+0000","last_fullsized":"2023-07-06T21:40:54.668562+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:52.030881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036427+0000","last_change":"2023-07-06T21:40:53.192677+0000","last_active":"2023-07-06T21:40:56.036427+0000","last_peered":"2023-07-06T21:40:56.036427+0000","last_clean":"2023-07-06T21:40:56.036427+0000","last_became_active":"2023-07-06T21:40:53.192014+0000","last_became_peered":"2023-07-06T21:40:53.192014+0000","last_unstale":"2023-07-06T21:40:56.036427+0000","last_undegraded":"2023-07-06T21:40:56.036427+0000","last_fullsized":"2023-07-06T21:40:56.036427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:39:24.803257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482739+0000","last_change":"2023-07-06T21:40:53.681150+0000","last_active":"2023-07-06T21:40:56.482739+0000","last_peered":"2023-07-06T21:40:56.482739+0000","last_clean":"2023-07-06T21:40:56.482739+0000","last_became_active":"2023-07-06T21:40:53.680511+0000","last_became_peered":"2023-07-06T21:40:53.680511+0000","last_unstale":"2023-07-06T21:40:56.482739+0000","last_undegraded":"2023-07-06T21:40:56.482739+0000","last_fullsized":"2023-07-06T21:40:56.482739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:48:25.388658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381686+0000","last_change":"2023-07-06T21:40:53.187483+0000","last_active":"2023-07-06T21:40:55.381686+0000","last_peered":"2023-07-06T21:40:55.381686+0000","last_clean":"2023-07-06T21:40:55.381686+0000","last_became_active":"2023-07-06T21:40:53.187095+0000","last_became_peered":"2023-07-06T21:40:53.187095+0000","last_unstale":"2023-07-06T21:40:55.381686+0000","last_undegraded":"2023-07-06T21:40:55.381686+0000","last_fullsized":"2023-07-06T21:40:55.381686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:45:15.764685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684219+0000","last_change":"2023-07-06T21:40:53.684178+0000","last_active":"2023-07-06T21:40:55.684219+0000","last_peered":"2023-07-06T21:40:55.684219+0000","last_clean":"2023-07-06T21:40:55.684219+0000","last_became_active":"2023-07-06T21:40:53.683883+0000","last_became_peered":"2023-07-06T21:40:53.683883+0000","last_unstale":"2023-07-06T21:40:55.684219+0000","last_undegraded":"2023-07-06T21:40:55.684219+0000","last_fullsized":"2023-07-06T21:40:55.684219+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:19:34.782540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482526+0000","last_change":"2023-07-06T21:40:53.182219+0000","last_active":"2023-07-06T21:40:56.482526+0000","last_peered":"2023-07-06T21:40:56.482526+0000","last_clean":"2023-07-06T21:40:56.482526+0000","last_became_active":"2023-07-06T21:40:53.181951+0000","last_became_peered":"2023-07-06T21:40:53.181951+0000","last_unstale":"2023-07-06T21:40:56.482526+0000","last_undegraded":"2023-07-06T21:40:56.482526+0000","last_fullsized":"2023-07-06T21:40:56.482526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:23.388862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664338+0000","last_change":"2023-07-06T21:40:53.672834+0000","last_active":"2023-07-06T21:40:54.664338+0000","last_peered":"2023-07-06T21:40:54.664338+0000","last_clean":"2023-07-06T21:40:54.664338+0000","last_became_active":"2023-07-06T21:40:53.672525+0000","last_became_peered":"2023-07-06T21:40:53.672525+0000","last_unstale":"2023-07-06T21:40:54.664338+0000","last_undegraded":"2023-07-06T21:40:54.664338+0000","last_fullsized":"2023-07-06T21:40:54.664338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:13:27.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684335+0000","last_change":"2023-07-06T21:40:53.188952+0000","last_active":"2023-07-06T21:40:55.684335+0000","last_peered":"2023-07-06T21:40:55.684335+0000","last_clean":"2023-07-06T21:40:55.684335+0000","last_became_active":"2023-07-06T21:40:53.188701+0000","last_became_peered":"2023-07-06T21:40:53.188701+0000","last_unstale":"2023-07-06T21:40:55.684335+0000","last_undegraded":"2023-07-06T21:40:55.684335+0000","last_fullsized":"2023-07-06T21:40:55.684335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:09:04.945994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036797+0000","last_change":"2023-07-06T21:40:53.693391+0000","last_active":"2023-07-06T21:40:56.036797+0000","last_peered":"2023-07-06T21:40:56.036797+0000","last_clean":"2023-07-06T21:40:56.036797+0000","last_became_active":"2023-07-06T21:40:53.692733+0000","last_became_peered":"2023-07-06T21:40:53.692733+0000","last_unstale":"2023-07-06T21:40:56.036797+0000","last_undegraded":"2023-07-06T21:40:56.036797+0000","last_fullsized":"2023-07-06T21:40:56.036797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:03:53.052919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668563+0000","last_change":"2023-07-06T21:40:53.187592+0000","last_active":"2023-07-06T21:40:54.668563+0000","last_peered":"2023-07-06T21:40:54.668563+0000","last_clean":"2023-07-06T21:40:54.668563+0000","last_became_active":"2023-07-06T21:40:53.187241+0000","last_became_peered":"2023-07-06T21:40:53.187241+0000","last_unstale":"2023-07-06T21:40:54.668563+0000","last_undegraded":"2023-07-06T21:40:54.668563+0000","last_fullsized":"2023-07-06T21:40:54.668563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:58:34.089052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482207+0000","last_change":"2023-07-06T21:40:53.689735+0000","last_active":"2023-07-06T21:40:56.482207+0000","last_peered":"2023-07-06T21:40:56.482207+0000","last_clean":"2023-07-06T21:40:56.482207+0000","last_became_active":"2023-07-06T21:40:53.689118+0000","last_became_peered":"2023-07-06T21:40:53.689118+0000","last_unstale":"2023-07-06T21:40:56.482207+0000","last_undegraded":"2023-07-06T21:40:56.482207+0000","last_fullsized":"2023-07-06T21:40:56.482207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:43:54.902088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668401+0000","last_change":"2023-07-06T21:40:53.183990+0000","last_active":"2023-07-06T21:40:54.668401+0000","last_peered":"2023-07-06T21:40:54.668401+0000","last_clean":"2023-07-06T21:40:54.668401+0000","last_became_active":"2023-07-06T21:40:53.183694+0000","last_became_peered":"2023-07-06T21:40:53.183694+0000","last_unstale":"2023-07-06T21:40:54.668401+0000","last_undegraded":"2023-07-06T21:40:54.668401+0000","last_fullsized":"2023-07-06T21:40:54.668401+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:48:24.380571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483074+0000","last_change":"2023-07-06T21:40:53.680883+0000","last_active":"2023-07-06T21:40:56.483074+0000","last_peered":"2023-07-06T21:40:56.483074+0000","last_clean":"2023-07-06T21:40:56.483074+0000","last_became_active":"2023-07-06T21:40:53.680527+0000","last_became_peered":"2023-07-06T21:40:53.680527+0000","last_unstale":"2023-07-06T21:40:56.483074+0000","last_undegraded":"2023-07-06T21:40:56.483074+0000","last_fullsized":"2023-07-06T21:40:56.483074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:58:35.097140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684171+0000","last_change":"2023-07-06T21:40:53.190453+0000","last_active":"2023-07-06T21:40:55.684171+0000","last_peered":"2023-07-06T21:40:55.684171+0000","last_clean":"2023-07-06T21:40:55.684171+0000","last_became_active":"2023-07-06T21:40:53.190201+0000","last_became_peered":"2023-07-06T21:40:53.190201+0000","last_unstale":"2023-07-06T21:40:55.684171+0000","last_undegraded":"2023-07-06T21:40:55.684171+0000","last_fullsized":"2023-07-06T21:40:55.684171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:40:25.368063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482035+0000","last_change":"2023-07-06T21:40:53.690053+0000","last_active":"2023-07-06T21:40:56.482035+0000","last_peered":"2023-07-06T21:40:56.482035+0000","last_clean":"2023-07-06T21:40:56.482035+0000","last_became_active":"2023-07-06T21:40:53.689804+0000","last_became_peered":"2023-07-06T21:40:53.689804+0000","last_unstale":"2023-07-06T21:40:56.482035+0000","last_undegraded":"2023-07-06T21:40:56.482035+0000","last_fullsized":"2023-07-06T21:40:56.482035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:08.486027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684240+0000","last_change":"2023-07-06T21:40:53.187472+0000","last_active":"2023-07-06T21:40:55.684240+0000","last_peered":"2023-07-06T21:40:55.684240+0000","last_clean":"2023-07-06T21:40:55.684240+0000","last_became_active":"2023-07-06T21:40:53.187247+0000","last_became_peered":"2023-07-06T21:40:53.187247+0000","last_unstale":"2023-07-06T21:40:55.684240+0000","last_undegraded":"2023-07-06T21:40:55.684240+0000","last_fullsized":"2023-07-06T21:40:55.684240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:01:32.938122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.683877+0000","last_change":"2023-07-06T21:40:53.682812+0000","last_active":"2023-07-06T21:40:55.683877+0000","last_peered":"2023-07-06T21:40:55.683877+0000","last_clean":"2023-07-06T21:40:55.683877+0000","last_became_active":"2023-07-06T21:40:53.682518+0000","last_became_peered":"2023-07-06T21:40:53.682518+0000","last_unstale":"2023-07-06T21:40:55.683877+0000","last_undegraded":"2023-07-06T21:40:55.683877+0000","last_fullsized":"2023-07-06T21:40:55.683877+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:28.083865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381580+0000","last_change":"2023-07-06T21:40:53.186493+0000","last_active":"2023-07-06T21:40:55.381580+0000","last_peered":"2023-07-06T21:40:55.381580+0000","last_clean":"2023-07-06T21:40:55.381580+0000","last_became_active":"2023-07-06T21:40:53.186188+0000","last_became_peered":"2023-07-06T21:40:53.186188+0000","last_unstale":"2023-07-06T21:40:55.381580+0000","last_undegraded":"2023-07-06T21:40:55.381580+0000","last_fullsized":"2023-07-06T21:40:55.381580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:39:13.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036670+0000","last_change":"2023-07-06T21:40:53.676345+0000","last_active":"2023-07-06T21:40:56.036670+0000","last_peered":"2023-07-06T21:40:56.036670+0000","last_clean":"2023-07-06T21:40:56.036670+0000","last_became_active":"2023-07-06T21:40:53.676040+0000","last_became_peered":"2023-07-06T21:40:53.676040+0000","last_unstale":"2023-07-06T21:40:56.036670+0000","last_undegraded":"2023-07-06T21:40:56.036670+0000","last_fullsized":"2023-07-06T21:40:56.036670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:31.464007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667398+0000","last_change":"2023-07-06T21:40:53.186727+0000","last_active":"2023-07-06T21:40:54.667398+0000","last_peered":"2023-07-06T21:40:54.667398+0000","last_clean":"2023-07-06T21:40:54.667398+0000","last_became_active":"2023-07-06T21:40:53.186335+0000","last_became_peered":"2023-07-06T21:40:53.186335+0000","last_unstale":"2023-07-06T21:40:54.667398+0000","last_undegraded":"2023-07-06T21:40:54.667398+0000","last_fullsized":"2023-07-06T21:40:54.667398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:58.270164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684008+0000","last_change":"2023-07-06T21:40:53.682830+0000","last_active":"2023-07-06T21:40:55.684008+0000","last_peered":"2023-07-06T21:40:55.684008+0000","last_clean":"2023-07-06T21:40:55.684008+0000","last_became_active":"2023-07-06T21:40:53.682587+0000","last_became_peered":"2023-07-06T21:40:53.682587+0000","last_unstale":"2023-07-06T21:40:55.684008+0000","last_undegraded":"2023-07-06T21:40:55.684008+0000","last_fullsized":"2023-07-06T21:40:55.684008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:52:48.741999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036794+0000","last_change":"2023-07-06T21:40:53.192564+0000","last_active":"2023-07-06T21:40:56.036794+0000","last_peered":"2023-07-06T21:40:56.036794+0000","last_clean":"2023-07-06T21:40:56.036794+0000","last_became_active":"2023-07-06T21:40:53.192281+0000","last_became_peered":"2023-07-06T21:40:53.192281+0000","last_unstale":"2023-07-06T21:40:56.036794+0000","last_undegraded":"2023-07-06T21:40:56.036794+0000","last_fullsized":"2023-07-06T21:40:56.036794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:10:57.566657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684330+0000","last_change":"2023-07-06T21:40:53.688292+0000","last_active":"2023-07-06T21:40:55.684330+0000","last_peered":"2023-07-06T21:40:55.684330+0000","last_clean":"2023-07-06T21:40:55.684330+0000","last_became_active":"2023-07-06T21:40:53.688047+0000","last_became_peered":"2023-07-06T21:40:53.688047+0000","last_unstale":"2023-07-06T21:40:55.684330+0000","last_undegraded":"2023-07-06T21:40:55.684330+0000","last_fullsized":"2023-07-06T21:40:55.684330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:41:19.252901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668649+0000","last_change":"2023-07-06T21:40:53.178836+0000","last_active":"2023-07-06T21:40:54.668649+0000","last_peered":"2023-07-06T21:40:54.668649+0000","last_clean":"2023-07-06T21:40:54.668649+0000","last_became_active":"2023-07-06T21:40:53.178562+0000","last_became_peered":"2023-07-06T21:40:53.178562+0000","last_unstale":"2023-07-06T21:40:54.668649+0000","last_undegraded":"2023-07-06T21:40:54.668649+0000","last_fullsized":"2023-07-06T21:40:54.668649+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:52:02.866459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668186+0000","last_change":"2023-07-06T21:40:53.686558+0000","last_active":"2023-07-06T21:40:54.668186+0000","last_peered":"2023-07-06T21:40:54.668186+0000","last_clean":"2023-07-06T21:40:54.668186+0000","last_became_active":"2023-07-06T21:40:53.686216+0000","last_became_peered":"2023-07-06T21:40:53.686216+0000","last_unstale":"2023-07-06T21:40:54.668186+0000","last_undegraded":"2023-07-06T21:40:54.668186+0000","last_fullsized":"2023-07-06T21:40:54.668186+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:35:46.789829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483153+0000","last_change":"2023-07-06T21:40:53.183012+0000","last_active":"2023-07-06T21:40:56.483153+0000","last_peered":"2023-07-06T21:40:56.483153+0000","last_clean":"2023-07-06T21:40:56.483153+0000","last_became_active":"2023-07-06T21:40:53.182747+0000","last_became_peered":"2023-07-06T21:40:53.182747+0000","last_unstale":"2023-07-06T21:40:56.483153+0000","last_undegraded":"2023-07-06T21:40:56.483153+0000","last_fullsized":"2023-07-06T21:40:56.483153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:48.463180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483131+0000","last_change":"2023-07-06T21:40:53.678992+0000","last_active":"2023-07-06T21:40:56.483131+0000","last_peered":"2023-07-06T21:40:56.483131+0000","last_clean":"2023-07-06T21:40:56.483131+0000","last_became_active":"2023-07-06T21:40:53.678689+0000","last_became_peered":"2023-07-06T21:40:53.678689+0000","last_unstale":"2023-07-06T21:40:56.483131+0000","last_undegraded":"2023-07-06T21:40:56.483131+0000","last_fullsized":"2023-07-06T21:40:56.483131+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:52:03.874547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664090+0000","last_change":"2023-07-06T21:40:53.189862+0000","last_active":"2023-07-06T21:40:54.664090+0000","last_peered":"2023-07-06T21:40:54.664090+0000","last_clean":"2023-07-06T21:40:54.664090+0000","last_became_active":"2023-07-06T21:40:53.189524+0000","last_became_peered":"2023-07-06T21:40:53.189524+0000","last_unstale":"2023-07-06T21:40:54.664090+0000","last_undegraded":"2023-07-06T21:40:54.664090+0000","last_fullsized":"2023-07-06T21:40:54.664090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:31:40.687007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664236+0000","last_change":"2023-07-06T21:40:53.678022+0000","last_active":"2023-07-06T21:40:54.664236+0000","last_peered":"2023-07-06T21:40:54.664236+0000","last_clean":"2023-07-06T21:40:54.664236+0000","last_became_active":"2023-07-06T21:40:53.677774+0000","last_became_peered":"2023-07-06T21:40:53.677774+0000","last_unstale":"2023-07-06T21:40:54.664236+0000","last_undegraded":"2023-07-06T21:40:54.664236+0000","last_fullsized":"2023-07-06T21:40:54.664236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:45:50.000441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381759+0000","last_change":"2023-07-06T21:40:53.185678+0000","last_active":"2023-07-06T21:40:55.381759+0000","last_peered":"2023-07-06T21:40:55.381759+0000","last_clean":"2023-07-06T21:40:55.381759+0000","last_became_active":"2023-07-06T21:40:53.185383+0000","last_became_peered":"2023-07-06T21:40:53.185383+0000","last_unstale":"2023-07-06T21:40:55.381759+0000","last_undegraded":"2023-07-06T21:40:55.381759+0000","last_fullsized":"2023-07-06T21:40:55.381759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:17:52.737294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036745+0000","last_change":"2023-07-06T21:40:53.696932+0000","last_active":"2023-07-06T21:40:56.036745+0000","last_peered":"2023-07-06T21:40:56.036745+0000","last_clean":"2023-07-06T21:40:56.036745+0000","last_became_active":"2023-07-06T21:40:53.689997+0000","last_became_peered":"2023-07-06T21:40:53.689997+0000","last_unstale":"2023-07-06T21:40:56.036745+0000","last_undegraded":"2023-07-06T21:40:56.036745+0000","last_fullsized":"2023-07-06T21:40:56.036745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:41:19.252901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663990+0000","last_change":"2023-07-06T21:40:53.179436+0000","last_active":"2023-07-06T21:40:54.663990+0000","last_peered":"2023-07-06T21:40:54.663990+0000","last_clean":"2023-07-06T21:40:54.663990+0000","last_became_active":"2023-07-06T21:40:53.179178+0000","last_became_peered":"2023-07-06T21:40:53.179178+0000","last_unstale":"2023-07-06T21:40:54.663990+0000","last_undegraded":"2023-07-06T21:40:54.663990+0000","last_fullsized":"2023-07-06T21:40:54.663990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:50:28.361917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.481979+0000","last_change":"2023-07-06T21:40:53.689719+0000","last_active":"2023-07-06T21:40:56.481979+0000","last_peered":"2023-07-06T21:40:56.481979+0000","last_clean":"2023-07-06T21:40:56.481979+0000","last_became_active":"2023-07-06T21:40:53.689430+0000","last_became_peered":"2023-07-06T21:40:53.689430+0000","last_unstale":"2023-07-06T21:40:56.481979+0000","last_undegraded":"2023-07-06T21:40:56.481979+0000","last_fullsized":"2023-07-06T21:40:56.481979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:35:04.612510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663962+0000","last_change":"2023-07-06T21:40:53.176020+0000","last_active":"2023-07-06T21:40:54.663962+0000","last_peered":"2023-07-06T21:40:54.663962+0000","last_clean":"2023-07-06T21:40:54.663962+0000","last_became_active":"2023-07-06T21:40:53.175748+0000","last_became_peered":"2023-07-06T21:40:53.175748+0000","last_unstale":"2023-07-06T21:40:54.663962+0000","last_undegraded":"2023-07-06T21:40:54.663962+0000","last_fullsized":"2023-07-06T21:40:54.663962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:58.270164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667500+0000","last_change":"2023-07-06T21:40:53.692043+0000","last_active":"2023-07-06T21:40:54.667500+0000","last_peered":"2023-07-06T21:40:54.667500+0000","last_clean":"2023-07-06T21:40:54.667500+0000","last_became_active":"2023-07-06T21:40:53.691745+0000","last_became_peered":"2023-07-06T21:40:53.691745+0000","last_unstale":"2023-07-06T21:40:54.667500+0000","last_undegraded":"2023-07-06T21:40:54.667500+0000","last_fullsized":"2023-07-06T21:40:54.667500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:09:41.600615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036555+0000","last_change":"2023-07-06T21:40:53.189895+0000","last_active":"2023-07-06T21:40:56.036555+0000","last_peered":"2023-07-06T21:40:56.036555+0000","last_clean":"2023-07-06T21:40:56.036555+0000","last_became_active":"2023-07-06T21:40:53.189488+0000","last_became_peered":"2023-07-06T21:40:53.189488+0000","last_unstale":"2023-07-06T21:40:56.036555+0000","last_undegraded":"2023-07-06T21:40:56.036555+0000","last_fullsized":"2023-07-06T21:40:56.036555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:34:41.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684130+0000","last_change":"2023-07-06T21:40:53.684323+0000","last_active":"2023-07-06T21:40:55.684130+0000","last_peered":"2023-07-06T21:40:55.684130+0000","last_clean":"2023-07-06T21:40:55.684130+0000","last_became_active":"2023-07-06T21:40:53.684041+0000","last_became_peered":"2023-07-06T21:40:53.684041+0000","last_unstale":"2023-07-06T21:40:55.684130+0000","last_undegraded":"2023-07-06T21:40:55.684130+0000","last_fullsized":"2023-07-06T21:40:55.684130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:11.828608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663854+0000","last_change":"2023-07-06T21:40:53.181447+0000","last_active":"2023-07-06T21:40:54.663854+0000","last_peered":"2023-07-06T21:40:54.663854+0000","last_clean":"2023-07-06T21:40:54.663854+0000","last_became_active":"2023-07-06T21:40:53.181209+0000","last_became_peered":"2023-07-06T21:40:53.181209+0000","last_unstale":"2023-07-06T21:40:54.663854+0000","last_undegraded":"2023-07-06T21:40:54.663854+0000","last_fullsized":"2023-07-06T21:40:54.663854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:00:08.281271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.481867+0000","last_change":"2023-07-06T21:40:53.681810+0000","last_active":"2023-07-06T21:40:56.481867+0000","last_peered":"2023-07-06T21:40:56.481867+0000","last_clean":"2023-07-06T21:40:56.481867+0000","last_became_active":"2023-07-06T21:40:53.681490+0000","last_became_peered":"2023-07-06T21:40:53.681490+0000","last_unstale":"2023-07-06T21:40:56.481867+0000","last_undegraded":"2023-07-06T21:40:56.481867+0000","last_fullsized":"2023-07-06T21:40:56.481867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:36:09.578251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482064+0000","last_change":"2023-07-06T21:40:53.185048+0000","last_active":"2023-07-06T21:40:56.482064+0000","last_peered":"2023-07-06T21:40:56.482064+0000","last_clean":"2023-07-06T21:40:56.482064+0000","last_became_active":"2023-07-06T21:40:53.184786+0000","last_became_peered":"2023-07-06T21:40:53.184786+0000","last_unstale":"2023-07-06T21:40:56.482064+0000","last_undegraded":"2023-07-06T21:40:56.482064+0000","last_fullsized":"2023-07-06T21:40:56.482064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:47:57.673943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381881+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:40:55.381881+0000","last_peered":"2023-07-06T21:40:55.381881+0000","last_clean":"2023-07-06T21:40:55.381881+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:40:55.381881+0000","last_undegraded":"2023-07-06T21:40:55.381881+0000","last_fullsized":"2023-07-06T21:40:55.381881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:39:45.590139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684196+0000","last_change":"2023-07-06T21:40:53.677480+0000","last_active":"2023-07-06T21:40:55.684196+0000","last_peered":"2023-07-06T21:40:55.684196+0000","last_clean":"2023-07-06T21:40:55.684196+0000","last_became_active":"2023-07-06T21:40:53.677213+0000","last_became_peered":"2023-07-06T21:40:53.677213+0000","last_unstale":"2023-07-06T21:40:55.684196+0000","last_undegraded":"2023-07-06T21:40:55.684196+0000","last_fullsized":"2023-07-06T21:40:55.684196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:45:16.772773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482185+0000","last_change":"2023-07-06T21:40:53.183577+0000","last_active":"2023-07-06T21:40:56.482185+0000","last_peered":"2023-07-06T21:40:56.482185+0000","last_clean":"2023-07-06T21:40:56.482185+0000","last_became_active":"2023-07-06T21:40:53.183328+0000","last_became_peered":"2023-07-06T21:40:53.183328+0000","last_unstale":"2023-07-06T21:40:56.482185+0000","last_undegraded":"2023-07-06T21:40:56.482185+0000","last_fullsized":"2023-07-06T21:40:56.482185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:30.938062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668415+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:40:54.668415+0000","last_peered":"2023-07-06T21:40:54.668415+0000","last_clean":"2023-07-06T21:40:54.668415+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:40:54.668415+0000","last_undegraded":"2023-07-06T21:40:54.668415+0000","last_fullsized":"2023-07-06T21:40:54.668415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:21:41.048106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036908+0000","last_change":"2023-07-06T21:40:53.693490+0000","last_active":"2023-07-06T21:40:56.036908+0000","last_peered":"2023-07-06T21:40:56.036908+0000","last_clean":"2023-07-06T21:40:56.036908+0000","last_became_active":"2023-07-06T21:40:53.693146+0000","last_became_peered":"2023-07-06T21:40:53.693146+0000","last_unstale":"2023-07-06T21:40:56.036908+0000","last_undegraded":"2023-07-06T21:40:56.036908+0000","last_fullsized":"2023-07-06T21:40:56.036908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:37.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664142+0000","last_change":"2023-07-06T21:40:53.175426+0000","last_active":"2023-07-06T21:40:54.664142+0000","last_peered":"2023-07-06T21:40:54.664142+0000","last_clean":"2023-07-06T21:40:54.664142+0000","last_became_active":"2023-07-06T21:40:53.175023+0000","last_became_peered":"2023-07-06T21:40:53.175023+0000","last_unstale":"2023-07-06T21:40:54.664142+0000","last_undegraded":"2023-07-06T21:40:54.664142+0000","last_fullsized":"2023-07-06T21:40:54.664142+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:06:24.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663869+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:40:54.663869+0000","last_peered":"2023-07-06T21:40:54.663869+0000","last_clean":"2023-07-06T21:40:54.663869+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:40:54.663869+0000","last_undegraded":"2023-07-06T21:40:54.663869+0000","last_fullsized":"2023-07-06T21:40:54.663869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:16:23.533601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668503+0000","last_change":"2023-07-06T21:40:53.689180+0000","last_active":"2023-07-06T21:40:54.668503+0000","last_peered":"2023-07-06T21:40:54.668503+0000","last_clean":"2023-07-06T21:40:54.668503+0000","last_became_active":"2023-07-06T21:40:53.688790+0000","last_became_peered":"2023-07-06T21:40:53.688790+0000","last_unstale":"2023-07-06T21:40:54.668503+0000","last_undegraded":"2023-07-06T21:40:54.668503+0000","last_fullsized":"2023-07-06T21:40:54.668503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:02:53.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036523+0000","last_change":"2023-07-06T21:40:53.188502+0000","last_active":"2023-07-06T21:40:56.036523+0000","last_peered":"2023-07-06T21:40:56.036523+0000","last_clean":"2023-07-06T21:40:56.036523+0000","last_became_active":"2023-07-06T21:40:53.188181+0000","last_became_peered":"2023-07-06T21:40:53.188181+0000","last_unstale":"2023-07-06T21:40:56.036523+0000","last_undegraded":"2023-07-06T21:40:56.036523+0000","last_fullsized":"2023-07-06T21:40:56.036523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:09:40.592527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668402+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:40:54.668402+0000","last_peered":"2023-07-06T21:40:54.668402+0000","last_clean":"2023-07-06T21:40:54.668402+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:40:54.668402+0000","last_undegraded":"2023-07-06T21:40:54.668402+0000","last_fullsized":"2023-07-06T21:40:54.668402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:47.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036843+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:40:56.036843+0000","last_peered":"2023-07-06T21:40:56.036843+0000","last_clean":"2023-07-06T21:40:56.036843+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:40:56.036843+0000","last_undegraded":"2023-07-06T21:40:56.036843+0000","last_fullsized":"2023-07-06T21:40:56.036843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:43.581561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381776+0000","last_change":"2023-07-06T21:40:53.692690+0000","last_active":"2023-07-06T21:40:55.381776+0000","last_peered":"2023-07-06T21:40:55.381776+0000","last_clean":"2023-07-06T21:40:55.381776+0000","last_became_active":"2023-07-06T21:40:53.692398+0000","last_became_peered":"2023-07-06T21:40:53.692398+0000","last_unstale":"2023-07-06T21:40:55.381776+0000","last_undegraded":"2023-07-06T21:40:55.381776+0000","last_fullsized":"2023-07-06T21:40:55.381776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:19:34.782540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668650+0000","last_change":"2023-07-06T21:40:53.184527+0000","last_active":"2023-07-06T21:40:54.668650+0000","last_peered":"2023-07-06T21:40:54.668650+0000","last_clean":"2023-07-06T21:40:54.668650+0000","last_became_active":"2023-07-06T21:40:53.184194+0000","last_became_peered":"2023-07-06T21:40:53.184194+0000","last_unstale":"2023-07-06T21:40:54.668650+0000","last_undegraded":"2023-07-06T21:40:54.668650+0000","last_fullsized":"2023-07-06T21:40:54.668650+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:27.793540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036974+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:40:56.036974+0000","last_peered":"2023-07-06T21:40:56.036974+0000","last_clean":"2023-07-06T21:40:56.036974+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:40:56.036974+0000","last_undegraded":"2023-07-06T21:40:56.036974+0000","last_fullsized":"2023-07-06T21:40:56.036974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:46:51.725897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381516+0000","last_change":"2023-07-06T21:40:53.685517+0000","last_active":"2023-07-06T21:40:55.381516+0000","last_peered":"2023-07-06T21:40:55.381516+0000","last_clean":"2023-07-06T21:40:55.381516+0000","last_became_active":"2023-07-06T21:40:53.685054+0000","last_became_peered":"2023-07-06T21:40:53.685054+0000","last_unstale":"2023-07-06T21:40:55.381516+0000","last_undegraded":"2023-07-06T21:40:55.381516+0000","last_fullsized":"2023-07-06T21:40:55.381516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:34:05.003871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667782+0000","last_change":"2023-07-06T21:40:53.183316+0000","last_active":"2023-07-06T21:40:54.667782+0000","last_peered":"2023-07-06T21:40:54.667782+0000","last_clean":"2023-07-06T21:40:54.667782+0000","last_became_active":"2023-07-06T21:40:53.182240+0000","last_became_peered":"2023-07-06T21:40:53.182240+0000","last_unstale":"2023-07-06T21:40:54.667782+0000","last_undegraded":"2023-07-06T21:40:54.667782+0000","last_fullsized":"2023-07-06T21:40:54.667782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:27:16.473263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036958+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:40:56.036958+0000","last_peered":"2023-07-06T21:40:56.036958+0000","last_clean":"2023-07-06T21:40:56.036958+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:40:56.036958+0000","last_undegraded":"2023-07-06T21:40:56.036958+0000","last_fullsized":"2023-07-06T21:40:56.036958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:47.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482889+0000","last_change":"2023-07-06T21:40:53.677627+0000","last_active":"2023-07-06T21:40:56.482889+0000","last_peered":"2023-07-06T21:40:56.482889+0000","last_clean":"2023-07-06T21:40:56.482889+0000","last_became_active":"2023-07-06T21:40:53.677341+0000","last_became_peered":"2023-07-06T21:40:53.677341+0000","last_unstale":"2023-07-06T21:40:56.482889+0000","last_undegraded":"2023-07-06T21:40:56.482889+0000","last_fullsized":"2023-07-06T21:40:56.482889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:23:14.710868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482609+0000","last_change":"2023-07-06T21:40:53.189329+0000","last_active":"2023-07-06T21:40:56.482609+0000","last_peered":"2023-07-06T21:40:56.482609+0000","last_clean":"2023-07-06T21:40:56.482609+0000","last_became_active":"2023-07-06T21:40:53.189041+0000","last_became_peered":"2023-07-06T21:40:53.189041+0000","last_unstale":"2023-07-06T21:40:56.482609+0000","last_undegraded":"2023-07-06T21:40:56.482609+0000","last_fullsized":"2023-07-06T21:40:56.482609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:40:19.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381700+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:40:55.381700+0000","last_peered":"2023-07-06T21:40:55.381700+0000","last_clean":"2023-07-06T21:40:55.381700+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:40:55.381700+0000","last_undegraded":"2023-07-06T21:40:55.381700+0000","last_fullsized":"2023-07-06T21:40:55.381700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:52.713389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036857+0000","last_change":"2023-07-06T21:40:53.686880+0000","last_active":"2023-07-06T21:40:56.036857+0000","last_peered":"2023-07-06T21:40:56.036857+0000","last_clean":"2023-07-06T21:40:56.036857+0000","last_became_active":"2023-07-06T21:40:53.686581+0000","last_became_peered":"2023-07-06T21:40:53.686581+0000","last_unstale":"2023-07-06T21:40:56.036857+0000","last_undegraded":"2023-07-06T21:40:56.036857+0000","last_fullsized":"2023-07-06T21:40:56.036857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:22.655067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.683887+0000","last_change":"2023-07-06T21:40:53.184306+0000","last_active":"2023-07-06T21:40:55.683887+0000","last_peered":"2023-07-06T21:40:55.683887+0000","last_clean":"2023-07-06T21:40:55.683887+0000","last_became_active":"2023-07-06T21:40:53.184027+0000","last_became_peered":"2023-07-06T21:40:53.184027+0000","last_unstale":"2023-07-06T21:40:55.683887+0000","last_undegraded":"2023-07-06T21:40:55.683887+0000","last_fullsized":"2023-07-06T21:40:55.683887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:06.526265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668628+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:40:54.668628+0000","last_peered":"2023-07-06T21:40:54.668628+0000","last_clean":"2023-07-06T21:40:54.668628+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:40:54.668628+0000","last_undegraded":"2023-07-06T21:40:54.668628+0000","last_fullsized":"2023-07-06T21:40:54.668628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:43.505095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.97599999999999998}]},{"osd":6,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.996}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.6779999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.6200000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.875}]},{"osd":4,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.86199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.5129999999999999,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.442}]},{"osd":2,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":11.913,"5min":11.913,"15min":11.913},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":22.053000000000001,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":9.1920000000000002,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.622}]},{"osd":5,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.58199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":1015808,"data_stored":764771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.433}]},{"osd":2,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.377},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.34300000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.5859999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.367,"5min":2.367,"15min":2.367},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.502}]},{"osd":4,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.621}]},{"osd":5,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":176986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.86799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":1.871}]},{"osd":1,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.4650000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.375},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.3100000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":2.0550000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.431}]},{"osd":5,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.929}]},{"osd":6,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":2.859},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.88}]},{"osd":7,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":2.1339999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":2.2410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.71899999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.96899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:40:58.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:40:58.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.561+0000 7f1890dff700 1 Processor -- start 2023-07-06T21:40:58.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.562+0000 7f1890dff700 1 -- start start 2023-07-06T21:40:58.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1890dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f188c15c6a0 0x7f188c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1890dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c06b8a0 con 0x7f188c0b3010 2023-07-06T21:40:58.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1890dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c06bb80 con 0x7f188c0b2470 2023-07-06T21:40:58.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1890dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c06b6b0 con 0x7f188c15c6a0 2023-07-06T21:40:58.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1883fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f188c0b3010 0x7f188c06afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48316/0 (socket says 172.21.15.84:48316) 2023-07-06T21:40:58.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1883fff700 1 -- 172.21.15.84:0/3954255224 learned_addr learned my addr 172.21.15.84:0/3954255224 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:58.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f188a59c700 1 --2- 172.21.15.84:0/3954255224 >> v2:172.21.15.84:3300/0 conn(0x7f188c15c6a0 0x7f188c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1889d9b700 1 -- 172.21.15.84:0/3954255224 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711977772 0 0) 0x7f188c06b8a0 con 0x7f188c0b3010 2023-07-06T21:40:58.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.563+0000 7f1889d9b700 1 -- 172.21.15.84:0/3954255224 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18780035b0 con 0x7f188c0b3010 2023-07-06T21:40:58.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f188a59c700 1 -- 172.21.15.84:0/3954255224 >> v1:172.21.15.107:6789/0 conn(0x7f188c0b2470 legacy=0x7f188c0cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:58.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f188a59c700 1 -- 172.21.15.84:0/3954255224 >> v1:172.21.15.84:6789/0 conn(0x7f188c0b3010 legacy=0x7f188c06afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f188a59c700 1 -- 172.21.15.84:0/3954255224 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c06be60 con 0x7f188c15c6a0 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f188a59c700 1 --2- 172.21.15.84:0/3954255224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c06c940 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f187c009840 tx=0x7f187c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8c048eb0d3ff5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f1889d9b700 1 -- 172.21.15.84:0/3954255224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c004230 con 0x7f188c15c6a0 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f1889d9b700 1 -- 172.21.15.84:0/3954255224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f187c004390 con 0x7f188c15c6a0 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.564+0000 7f1889d9b700 1 -- 172.21.15.84:0/3954255224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f187c0036c0 con 0x7f188c15c6a0 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 -- 172.21.15.84:0/3954255224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 msgr2=0x7f188c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 --2- 172.21.15.84:0/3954255224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c06c940 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f187c009840 tx=0x7f187c009b50 comp rx=0 tx=0).stop 2023-07-06T21:40:58.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 -- 172.21.15.84:0/3954255224 shutdown_connections 2023-07-06T21:40:58.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 --2- 172.21.15.84:0/3954255224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c06c940 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 -- 172.21.15.84:0/3954255224 >> 172.21.15.84:0/3954255224 conn(0x7f188c06e640 msgr2=0x7f188c06ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:58.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 -- 172.21.15.84:0/3954255224 shutdown_connections 2023-07-06T21:40:58.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.565+0000 7f1890dff700 1 -- 172.21.15.84:0/3954255224 wait complete. 2023-07-06T21:40:58.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.567+0000 7f1890dff700 1 Processor -- start 2023-07-06T21:40:58.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.567+0000 7f1890dff700 1 -- start start 2023-07-06T21:40:58.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.568+0000 7f1890dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f188c0b2470 0x7f188c0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.568+0000 7f1890dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f188c0b3010 0x7f188c15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f1890dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f1890dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c97c0 con 0x7f188c15c6a0 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f1890dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c8bb0 con 0x7f188c0b2470 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f1890dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0c8e90 con 0x7f188c0b3010 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53244/0 (socket says 172.21.15.84:53244) 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 -- 172.21.15.84:0/2802675161 learned_addr learned my addr 172.21.15.84:0/2802675161 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f1883fff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f188c0b3010 0x7f188c15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188a59c700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f188c0b2470 0x7f188c0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 -- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f188c0b3010 msgr2=0x7f188c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f188c0b3010 0x7f188c15f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 -- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f188c0b2470 msgr2=0x7f188c0b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f188c0b2470 0x7f188c0b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 -- 172.21.15.84:0/2802675161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f187c0094a0 con 0x7f188c15c6a0 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f1883fff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f188c0b3010 0x7f188c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188a59c700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f188c0b2470 0x7f188c0b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:58.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.569+0000 7f188ad9d700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c0c9c50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f188400cc10 tx=0x7f188400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:58.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.570+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188400abb0 con 0x7f188c15c6a0 2023-07-06T21:40:58.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.570+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188400ad10 con 0x7f188c15c6a0 2023-07-06T21:40:58.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.570+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1884012660 con 0x7f188c15c6a0 2023-07-06T21:40:58.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.570+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c0c9170 con 0x7f188c15c6a0 2023-07-06T21:40:58.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.570+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0b65a0 con 0x7f188c15c6a0 2023-07-06T21:40:58.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.571+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c0cf1f0 con 0x7f188c15c6a0 2023-07-06T21:40:58.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.572+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f18840127c0 con 0x7f188c15c6a0 2023-07-06T21:40:58.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.573+0000 7f1882ffd700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f186c06ace0 0x7f186c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:58.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.573+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1884089c10 con 0x7f188c15c6a0 2023-07-06T21:40:58.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.573+0000 7f188a59c700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f186c06ace0 0x7f186c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:58.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.573+0000 7f188a59c700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f186c06ace0 0x7f186c06d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f187c0090b0 tx=0x7f187c010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:58.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.577+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1884054cb0 con 0x7f188c15c6a0 2023-07-06T21:40:58.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.770+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f188c06c420 con 0x7f186c06ace0 2023-07-06T21:40:58.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.779+0000 7f1882ffd700 1 -- 172.21.15.84:0/2802675161 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311157 (secure 0 0 0) 0x7f187c0036a0 con 0x7f186c06ace0 2023-07-06T21:40:58.786 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:58.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.783+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f186c06ace0 msgr2=0x7f186c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.783+0000 7f1890dff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f186c06ace0 0x7f186c06d190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f187c0090b0 tx=0x7f187c010000 comp rx=0 tx=0).stop 2023-07-06T21:40:58.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.783+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 msgr2=0x7f188c0c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:58.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.783+0000 7f1890dff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c0c9c50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f188400cc10 tx=0x7f188400a060 comp rx=0 tx=0).stop 2023-07-06T21:40:58.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 shutdown_connections 2023-07-06T21:40:58.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f186c06ace0 0x7f186c06d190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f188c15c6a0 0x7f188c0c9c50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f188c0b2470 0x7f188c0b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 --2- 172.21.15.84:0/2802675161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f188c0b3010 0x7f188c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:58.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 >> 172.21.15.84:0/2802675161 conn(0x7f188c06e640 msgr2=0x7f188c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:58.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 shutdown_connections 2023-07-06T21:40:58.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:58.784+0000 7f1890dff700 1 -- 172.21.15.84:0/2802675161 wait complete. 2023-07-06T21:40:58.792 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:40:58.804 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-07-06T21:40:58.154293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220132,"kb_used_data":4516,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754754588,"statfs":{"total":773094113280,"available":772868698112,"internally_reserved":0,"allocated":4624384,"data_stored":2587545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.991414"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684248+0000","last_change":"2023-07-06T21:40:53.685275+0000","last_active":"2023-07-06T21:40:55.684248+0000","last_peered":"2023-07-06T21:40:55.684248+0000","last_clean":"2023-07-06T21:40:55.684248+0000","last_became_active":"2023-07-06T21:40:53.685007+0000","last_became_peered":"2023-07-06T21:40:53.685007+0000","last_unstale":"2023-07-06T21:40:55.684248+0000","last_undegraded":"2023-07-06T21:40:55.684248+0000","last_fullsized":"2023-07-06T21:40:55.684248+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:17:53.745382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684199+0000","last_change":"2023-07-06T21:40:53.688448+0000","last_active":"2023-07-06T21:40:55.684199+0000","last_peered":"2023-07-06T21:40:55.684199+0000","last_clean":"2023-07-06T21:40:55.684199+0000","last_became_active":"2023-07-06T21:40:53.688150+0000","last_became_peered":"2023-07-06T21:40:53.688150+0000","last_unstale":"2023-07-06T21:40:55.684199+0000","last_undegraded":"2023-07-06T21:40:55.684199+0000","last_fullsized":"2023-07-06T21:40:55.684199+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:39:14.937173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482163+0000","last_change":"2023-07-06T21:40:53.691316+0000","last_active":"2023-07-06T21:40:56.482163+0000","last_peered":"2023-07-06T21:40:56.482163+0000","last_clean":"2023-07-06T21:40:56.482163+0000","last_became_active":"2023-07-06T21:40:53.691051+0000","last_became_peered":"2023-07-06T21:40:53.691051+0000","last_unstale":"2023-07-06T21:40:56.482163+0000","last_undegraded":"2023-07-06T21:40:56.482163+0000","last_fullsized":"2023-07-06T21:40:56.482163+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:04.716858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664295+0000","last_change":"2023-07-06T21:40:53.672857+0000","last_active":"2023-07-06T21:40:54.664295+0000","last_peered":"2023-07-06T21:40:54.664295+0000","last_clean":"2023-07-06T21:40:54.664295+0000","last_became_active":"2023-07-06T21:40:53.672557+0000","last_became_peered":"2023-07-06T21:40:53.672557+0000","last_unstale":"2023-07-06T21:40:54.664295+0000","last_undegraded":"2023-07-06T21:40:54.664295+0000","last_fullsized":"2023-07-06T21:40:54.664295+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:41:46.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664230+0000","last_change":"2023-07-06T21:40:53.671963+0000","last_active":"2023-07-06T21:40:54.664230+0000","last_peered":"2023-07-06T21:40:54.664230+0000","last_clean":"2023-07-06T21:40:54.664230+0000","last_became_active":"2023-07-06T21:40:53.671695+0000","last_became_peered":"2023-07-06T21:40:53.671695+0000","last_unstale":"2023-07-06T21:40:54.664230+0000","last_undegraded":"2023-07-06T21:40:54.664230+0000","last_fullsized":"2023-07-06T21:40:54.664230+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:21:02.102040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381518+0000","last_change":"2023-07-06T21:40:53.678253+0000","last_active":"2023-07-06T21:40:55.381518+0000","last_peered":"2023-07-06T21:40:55.381518+0000","last_clean":"2023-07-06T21:40:55.381518+0000","last_became_active":"2023-07-06T21:40:53.677998+0000","last_became_peered":"2023-07-06T21:40:53.677998+0000","last_unstale":"2023-07-06T21:40:55.381518+0000","last_undegraded":"2023-07-06T21:40:55.381518+0000","last_fullsized":"2023-07-06T21:40:55.381518+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:07.534353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668015+0000","last_change":"2023-07-06T21:40:53.691770+0000","last_active":"2023-07-06T21:40:54.668015+0000","last_peered":"2023-07-06T21:40:54.668015+0000","last_clean":"2023-07-06T21:40:54.668015+0000","last_became_active":"2023-07-06T21:40:53.689786+0000","last_became_peered":"2023-07-06T21:40:53.689786+0000","last_unstale":"2023-07-06T21:40:54.668015+0000","last_undegraded":"2023-07-06T21:40:54.668015+0000","last_fullsized":"2023-07-06T21:40:54.668015+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:34:42.229390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.683876+0000","last_change":"2023-07-06T21:40:53.681428+0000","last_active":"2023-07-06T21:40:55.683876+0000","last_peered":"2023-07-06T21:40:55.683876+0000","last_clean":"2023-07-06T21:40:55.683876+0000","last_became_active":"2023-07-06T21:40:53.681141+0000","last_became_peered":"2023-07-06T21:40:53.681141+0000","last_unstale":"2023-07-06T21:40:55.683876+0000","last_undegraded":"2023-07-06T21:40:55.683876+0000","last_fullsized":"2023-07-06T21:40:55.683876+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:34:05.003871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482299+0000","last_change":"2023-07-06T21:40:53.689527+0000","last_active":"2023-07-06T21:40:56.482299+0000","last_peered":"2023-07-06T21:40:56.482299+0000","last_clean":"2023-07-06T21:40:56.482299+0000","last_became_active":"2023-07-06T21:40:53.689216+0000","last_became_peered":"2023-07-06T21:40:53.689216+0000","last_unstale":"2023-07-06T21:40:56.482299+0000","last_undegraded":"2023-07-06T21:40:56.482299+0000","last_fullsized":"2023-07-06T21:40:56.482299+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:54.151894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668381+0000","last_change":"2023-07-06T21:40:53.692783+0000","last_active":"2023-07-06T21:40:54.668381+0000","last_peered":"2023-07-06T21:40:54.668381+0000","last_clean":"2023-07-06T21:40:54.668381+0000","last_became_active":"2023-07-06T21:40:53.692496+0000","last_became_peered":"2023-07-06T21:40:53.692496+0000","last_unstale":"2023-07-06T21:40:54.668381+0000","last_undegraded":"2023-07-06T21:40:54.668381+0000","last_fullsized":"2023-07-06T21:40:54.668381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:37.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668277+0000","last_change":"2023-07-06T21:40:53.688460+0000","last_active":"2023-07-06T21:40:54.668277+0000","last_peered":"2023-07-06T21:40:54.668277+0000","last_clean":"2023-07-06T21:40:54.668277+0000","last_became_active":"2023-07-06T21:40:53.688195+0000","last_became_peered":"2023-07-06T21:40:53.688195+0000","last_unstale":"2023-07-06T21:40:54.668277+0000","last_undegraded":"2023-07-06T21:40:54.668277+0000","last_fullsized":"2023-07-06T21:40:54.668277+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:22.655067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482066+0000","last_change":"2023-07-06T21:40:53.682035+0000","last_active":"2023-07-06T21:40:56.482066+0000","last_peered":"2023-07-06T21:40:56.482066+0000","last_clean":"2023-07-06T21:40:56.482066+0000","last_became_active":"2023-07-06T21:40:53.681671+0000","last_became_peered":"2023-07-06T21:40:53.681671+0000","last_unstale":"2023-07-06T21:40:56.482066+0000","last_undegraded":"2023-07-06T21:40:56.482066+0000","last_fullsized":"2023-07-06T21:40:56.482066+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:35:33.585902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668119+0000","last_change":"2023-07-06T21:40:53.692570+0000","last_active":"2023-07-06T21:40:54.668119+0000","last_peered":"2023-07-06T21:40:54.668119+0000","last_clean":"2023-07-06T21:40:54.668119+0000","last_became_active":"2023-07-06T21:40:53.692276+0000","last_became_peered":"2023-07-06T21:40:53.692276+0000","last_unstale":"2023-07-06T21:40:54.668119+0000","last_undegraded":"2023-07-06T21:40:54.668119+0000","last_fullsized":"2023-07-06T21:40:54.668119+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:03:53.052919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483096+0000","last_change":"2023-07-06T21:40:53.688112+0000","last_active":"2023-07-06T21:40:56.483096+0000","last_peered":"2023-07-06T21:40:56.483096+0000","last_clean":"2023-07-06T21:40:56.483096+0000","last_became_active":"2023-07-06T21:40:53.687820+0000","last_became_peered":"2023-07-06T21:40:53.687820+0000","last_unstale":"2023-07-06T21:40:56.483096+0000","last_undegraded":"2023-07-06T21:40:56.483096+0000","last_fullsized":"2023-07-06T21:40:56.483096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:28.801628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667836+0000","last_change":"2023-07-06T21:40:53.682885+0000","last_active":"2023-07-06T21:40:54.667836+0000","last_peered":"2023-07-06T21:40:54.667836+0000","last_clean":"2023-07-06T21:40:54.667836+0000","last_became_active":"2023-07-06T21:40:53.682538+0000","last_became_peered":"2023-07-06T21:40:53.682538+0000","last_unstale":"2023-07-06T21:40:54.667836+0000","last_undegraded":"2023-07-06T21:40:54.667836+0000","last_fullsized":"2023-07-06T21:40:54.667836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:31.464007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036552+0000","last_change":"2023-07-06T21:40:53.680201+0000","last_active":"2023-07-06T21:40:56.036552+0000","last_peered":"2023-07-06T21:40:56.036552+0000","last_clean":"2023-07-06T21:40:56.036552+0000","last_became_active":"2023-07-06T21:40:53.679930+0000","last_became_peered":"2023-07-06T21:40:53.679930+0000","last_unstale":"2023-07-06T21:40:56.036552+0000","last_undegraded":"2023-07-06T21:40:56.036552+0000","last_fullsized":"2023-07-06T21:40:56.036552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:01:33.946210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036773+0000","last_change":"2023-07-06T21:40:53.693208+0000","last_active":"2023-07-06T21:40:56.036773+0000","last_peered":"2023-07-06T21:40:56.036773+0000","last_clean":"2023-07-06T21:40:56.036773+0000","last_became_active":"2023-07-06T21:40:53.692337+0000","last_became_peered":"2023-07-06T21:40:53.692337+0000","last_unstale":"2023-07-06T21:40:56.036773+0000","last_undegraded":"2023-07-06T21:40:56.036773+0000","last_fullsized":"2023-07-06T21:40:56.036773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:35:46.789829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668442+0000","last_change":"2023-07-06T21:40:53.685670+0000","last_active":"2023-07-06T21:40:54.668442+0000","last_peered":"2023-07-06T21:40:54.668442+0000","last_clean":"2023-07-06T21:40:54.668442+0000","last_became_active":"2023-07-06T21:40:53.685349+0000","last_became_peered":"2023-07-06T21:40:53.685349+0000","last_unstale":"2023-07-06T21:40:54.668442+0000","last_undegraded":"2023-07-06T21:40:54.668442+0000","last_fullsized":"2023-07-06T21:40:54.668442+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:13:27.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664189+0000","last_change":"2023-07-06T21:40:53.683998+0000","last_active":"2023-07-06T21:40:54.664189+0000","last_peered":"2023-07-06T21:40:54.664189+0000","last_clean":"2023-07-06T21:40:54.664189+0000","last_became_active":"2023-07-06T21:40:53.683736+0000","last_became_peered":"2023-07-06T21:40:53.683736+0000","last_unstale":"2023-07-06T21:40:54.664189+0000","last_undegraded":"2023-07-06T21:40:54.664189+0000","last_fullsized":"2023-07-06T21:40:54.664189+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:27:17.481351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667941+0000","last_change":"2023-07-06T21:40:53.687772+0000","last_active":"2023-07-06T21:40:54.667941+0000","last_peered":"2023-07-06T21:40:54.667941+0000","last_clean":"2023-07-06T21:40:54.667941+0000","last_became_active":"2023-07-06T21:40:53.687283+0000","last_became_peered":"2023-07-06T21:40:53.687283+0000","last_unstale":"2023-07-06T21:40:54.667941+0000","last_undegraded":"2023-07-06T21:40:54.667941+0000","last_fullsized":"2023-07-06T21:40:54.667941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:45:50.000441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668298+0000","last_change":"2023-07-06T21:40:53.682118+0000","last_active":"2023-07-06T21:40:54.668298+0000","last_peered":"2023-07-06T21:40:54.668298+0000","last_clean":"2023-07-06T21:40:54.668298+0000","last_became_active":"2023-07-06T21:40:53.681788+0000","last_became_peered":"2023-07-06T21:40:53.681788+0000","last_unstale":"2023-07-06T21:40:54.668298+0000","last_undegraded":"2023-07-06T21:40:54.668298+0000","last_fullsized":"2023-07-06T21:40:54.668298+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:11:58.346462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482642+0000","last_change":"2023-07-06T21:40:53.681341+0000","last_active":"2023-07-06T21:40:56.482642+0000","last_peered":"2023-07-06T21:40:56.482642+0000","last_clean":"2023-07-06T21:40:56.482642+0000","last_became_active":"2023-07-06T21:40:53.680886+0000","last_became_peered":"2023-07-06T21:40:53.680886+0000","last_unstale":"2023-07-06T21:40:56.482642+0000","last_undegraded":"2023-07-06T21:40:56.482642+0000","last_fullsized":"2023-07-06T21:40:56.482642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:37.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036481+0000","last_change":"2023-07-06T21:40:53.688853+0000","last_active":"2023-07-06T21:40:56.036481+0000","last_peered":"2023-07-06T21:40:56.036481+0000","last_clean":"2023-07-06T21:40:56.036481+0000","last_became_active":"2023-07-06T21:40:53.688486+0000","last_became_peered":"2023-07-06T21:40:53.688486+0000","last_unstale":"2023-07-06T21:40:56.036481+0000","last_undegraded":"2023-07-06T21:40:56.036481+0000","last_fullsized":"2023-07-06T21:40:56.036481+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:09:05.954082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663893+0000","last_change":"2023-07-06T21:40:53.676302+0000","last_active":"2023-07-06T21:40:54.663893+0000","last_peered":"2023-07-06T21:40:54.663893+0000","last_clean":"2023-07-06T21:40:54.663893+0000","last_became_active":"2023-07-06T21:40:53.676007+0000","last_became_peered":"2023-07-06T21:40:53.676007+0000","last_unstale":"2023-07-06T21:40:54.663893+0000","last_undegraded":"2023-07-06T21:40:54.663893+0000","last_fullsized":"2023-07-06T21:40:54.663893+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:44:00.532865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668558+0000","last_change":"2023-07-06T21:40:53.685340+0000","last_active":"2023-07-06T21:40:54.668558+0000","last_peered":"2023-07-06T21:40:54.668558+0000","last_clean":"2023-07-06T21:40:54.668558+0000","last_became_active":"2023-07-06T21:40:53.685016+0000","last_became_peered":"2023-07-06T21:40:53.685016+0000","last_unstale":"2023-07-06T21:40:54.668558+0000","last_undegraded":"2023-07-06T21:40:54.668558+0000","last_fullsized":"2023-07-06T21:40:54.668558+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:06:42.264796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381914+0000","last_change":"2023-07-06T21:40:53.692733+0000","last_active":"2023-07-06T21:40:55.381914+0000","last_peered":"2023-07-06T21:40:55.381914+0000","last_clean":"2023-07-06T21:40:55.381914+0000","last_became_active":"2023-07-06T21:40:53.692433+0000","last_became_peered":"2023-07-06T21:40:53.692433+0000","last_unstale":"2023-07-06T21:40:55.381914+0000","last_undegraded":"2023-07-06T21:40:55.381914+0000","last_fullsized":"2023-07-06T21:40:55.381914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:11.828608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667780+0000","last_change":"2023-07-06T21:40:53.684666+0000","last_active":"2023-07-06T21:40:54.667780+0000","last_peered":"2023-07-06T21:40:54.667780+0000","last_clean":"2023-07-06T21:40:54.667780+0000","last_became_active":"2023-07-06T21:40:53.684371+0000","last_became_peered":"2023-07-06T21:40:53.684371+0000","last_unstale":"2023-07-06T21:40:54.667780+0000","last_undegraded":"2023-07-06T21:40:54.667780+0000","last_fullsized":"2023-07-06T21:40:54.667780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:31:41.695094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668337+0000","last_change":"2023-07-06T21:40:53.689521+0000","last_active":"2023-07-06T21:40:54.668337+0000","last_peered":"2023-07-06T21:40:54.668337+0000","last_clean":"2023-07-06T21:40:54.668337+0000","last_became_active":"2023-07-06T21:40:53.688866+0000","last_became_peered":"2023-07-06T21:40:53.688866+0000","last_unstale":"2023-07-06T21:40:54.668337+0000","last_undegraded":"2023-07-06T21:40:54.668337+0000","last_fullsized":"2023-07-06T21:40:54.668337+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:24.396950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668290+0000","last_change":"2023-07-06T21:40:53.685506+0000","last_active":"2023-07-06T21:40:54.668290+0000","last_peered":"2023-07-06T21:40:54.668290+0000","last_clean":"2023-07-06T21:40:54.668290+0000","last_became_active":"2023-07-06T21:40:53.685138+0000","last_became_peered":"2023-07-06T21:40:53.685138+0000","last_unstale":"2023-07-06T21:40:54.668290+0000","last_undegraded":"2023-07-06T21:40:54.668290+0000","last_fullsized":"2023-07-06T21:40:54.668290+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:41:46.662259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667807+0000","last_change":"2023-07-06T21:40:53.684696+0000","last_active":"2023-07-06T21:40:54.667807+0000","last_peered":"2023-07-06T21:40:54.667807+0000","last_clean":"2023-07-06T21:40:54.667807+0000","last_became_active":"2023-07-06T21:40:53.684366+0000","last_became_peered":"2023-07-06T21:40:53.684366+0000","last_unstale":"2023-07-06T21:40:54.667807+0000","last_undegraded":"2023-07-06T21:40:54.667807+0000","last_fullsized":"2023-07-06T21:40:54.667807+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:06:26.004386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036363+0000","last_change":"2023-07-06T21:40:53.678093+0000","last_active":"2023-07-06T21:40:56.036363+0000","last_peered":"2023-07-06T21:40:56.036363+0000","last_clean":"2023-07-06T21:40:56.036363+0000","last_became_active":"2023-07-06T21:40:53.677832+0000","last_became_peered":"2023-07-06T21:40:53.677832+0000","last_unstale":"2023-07-06T21:40:56.036363+0000","last_undegraded":"2023-07-06T21:40:56.036363+0000","last_fullsized":"2023-07-06T21:40:56.036363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:16:43.933175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381523+0000","last_change":"2023-07-06T21:40:53.681621+0000","last_active":"2023-07-06T21:40:55.381523+0000","last_peered":"2023-07-06T21:40:55.381523+0000","last_clean":"2023-07-06T21:40:55.381523+0000","last_became_active":"2023-07-06T21:40:53.681341+0000","last_became_peered":"2023-07-06T21:40:53.681341+0000","last_unstale":"2023-07-06T21:40:55.381523+0000","last_undegraded":"2023-07-06T21:40:55.381523+0000","last_fullsized":"2023-07-06T21:40:55.381523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:52:48.741999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036888+0000","last_change":"2023-07-06T21:40:53.692883+0000","last_active":"2023-07-06T21:40:56.036888+0000","last_peered":"2023-07-06T21:40:56.036888+0000","last_clean":"2023-07-06T21:40:56.036888+0000","last_became_active":"2023-07-06T21:40:53.692585+0000","last_became_peered":"2023-07-06T21:40:53.692585+0000","last_unstale":"2023-07-06T21:40:56.036888+0000","last_undegraded":"2023-07-06T21:40:56.036888+0000","last_fullsized":"2023-07-06T21:40:56.036888+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:11:58.346462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668531+0000","last_change":"2023-07-06T21:40:53.185932+0000","last_active":"2023-07-06T21:40:54.668531+0000","last_peered":"2023-07-06T21:40:54.668531+0000","last_clean":"2023-07-06T21:40:54.668531+0000","last_became_active":"2023-07-06T21:40:53.185631+0000","last_became_peered":"2023-07-06T21:40:53.185631+0000","last_unstale":"2023-07-06T21:40:54.668531+0000","last_undegraded":"2023-07-06T21:40:54.668531+0000","last_fullsized":"2023-07-06T21:40:54.668531+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:50:41.968014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381874+0000","last_change":"2023-07-06T21:40:53.693028+0000","last_active":"2023-07-06T21:40:55.381874+0000","last_peered":"2023-07-06T21:40:55.381874+0000","last_clean":"2023-07-06T21:40:55.381874+0000","last_became_active":"2023-07-06T21:40:53.692667+0000","last_became_peered":"2023-07-06T21:40:53.692667+0000","last_unstale":"2023-07-06T21:40:55.381874+0000","last_undegraded":"2023-07-06T21:40:55.381874+0000","last_fullsized":"2023-07-06T21:40:55.381874+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:01:33.946210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482185+0000","last_change":"2023-07-06T21:40:53.190299+0000","last_active":"2023-07-06T21:40:56.482185+0000","last_peered":"2023-07-06T21:40:56.482185+0000","last_clean":"2023-07-06T21:40:56.482185+0000","last_became_active":"2023-07-06T21:40:53.189951+0000","last_became_peered":"2023-07-06T21:40:53.189951+0000","last_unstale":"2023-07-06T21:40:56.482185+0000","last_undegraded":"2023-07-06T21:40:56.482185+0000","last_fullsized":"2023-07-06T21:40:56.482185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:09:14.716482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483056+0000","last_change":"2023-07-06T21:40:53.687852+0000","last_active":"2023-07-06T21:40:56.483056+0000","last_peered":"2023-07-06T21:40:56.483056+0000","last_clean":"2023-07-06T21:40:56.483056+0000","last_became_active":"2023-07-06T21:40:53.687513+0000","last_became_peered":"2023-07-06T21:40:53.687513+0000","last_unstale":"2023-07-06T21:40:56.483056+0000","last_undegraded":"2023-07-06T21:40:56.483056+0000","last_fullsized":"2023-07-06T21:40:56.483056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:52.030881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381892+0000","last_change":"2023-07-06T21:40:53.183298+0000","last_active":"2023-07-06T21:40:55.381892+0000","last_peered":"2023-07-06T21:40:55.381892+0000","last_clean":"2023-07-06T21:40:55.381892+0000","last_became_active":"2023-07-06T21:40:53.183047+0000","last_became_peered":"2023-07-06T21:40:53.183047+0000","last_unstale":"2023-07-06T21:40:55.381892+0000","last_undegraded":"2023-07-06T21:40:55.381892+0000","last_fullsized":"2023-07-06T21:40:55.381892+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:16:42.925087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684291+0000","last_change":"2023-07-06T21:40:53.687980+0000","last_active":"2023-07-06T21:40:55.684291+0000","last_peered":"2023-07-06T21:40:55.684291+0000","last_clean":"2023-07-06T21:40:55.684291+0000","last_became_active":"2023-07-06T21:40:53.687481+0000","last_became_peered":"2023-07-06T21:40:53.687481+0000","last_unstale":"2023-07-06T21:40:55.684291+0000","last_undegraded":"2023-07-06T21:40:55.684291+0000","last_fullsized":"2023-07-06T21:40:55.684291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:16:43.933175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482902+0000","last_change":"2023-07-06T21:40:53.187761+0000","last_active":"2023-07-06T21:40:56.482902+0000","last_peered":"2023-07-06T21:40:56.482902+0000","last_clean":"2023-07-06T21:40:56.482902+0000","last_became_active":"2023-07-06T21:40:53.187506+0000","last_became_peered":"2023-07-06T21:40:53.187506+0000","last_unstale":"2023-07-06T21:40:56.482902+0000","last_undegraded":"2023-07-06T21:40:56.482902+0000","last_fullsized":"2023-07-06T21:40:56.482902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:02:52.869881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.481828+0000","last_change":"2023-07-06T21:40:53.683553+0000","last_active":"2023-07-06T21:40:56.481828+0000","last_peered":"2023-07-06T21:40:56.481828+0000","last_clean":"2023-07-06T21:40:56.481828+0000","last_became_active":"2023-07-06T21:40:53.683289+0000","last_became_peered":"2023-07-06T21:40:53.683289+0000","last_unstale":"2023-07-06T21:40:56.481828+0000","last_undegraded":"2023-07-06T21:40:56.481828+0000","last_fullsized":"2023-07-06T21:40:56.481828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:49:18.670130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668232+0000","last_change":"2023-07-06T21:40:53.185220+0000","last_active":"2023-07-06T21:40:54.668232+0000","last_peered":"2023-07-06T21:40:54.668232+0000","last_clean":"2023-07-06T21:40:54.668232+0000","last_became_active":"2023-07-06T21:40:53.184950+0000","last_became_peered":"2023-07-06T21:40:53.184950+0000","last_unstale":"2023-07-06T21:40:54.668232+0000","last_undegraded":"2023-07-06T21:40:54.668232+0000","last_fullsized":"2023-07-06T21:40:54.668232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:10:57.566657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668562+0000","last_change":"2023-07-06T21:40:53.684111+0000","last_active":"2023-07-06T21:40:54.668562+0000","last_peered":"2023-07-06T21:40:54.668562+0000","last_clean":"2023-07-06T21:40:54.668562+0000","last_became_active":"2023-07-06T21:40:53.683833+0000","last_became_peered":"2023-07-06T21:40:53.683833+0000","last_unstale":"2023-07-06T21:40:54.668562+0000","last_undegraded":"2023-07-06T21:40:54.668562+0000","last_fullsized":"2023-07-06T21:40:54.668562+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:58:52.030881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036427+0000","last_change":"2023-07-06T21:40:53.192677+0000","last_active":"2023-07-06T21:40:56.036427+0000","last_peered":"2023-07-06T21:40:56.036427+0000","last_clean":"2023-07-06T21:40:56.036427+0000","last_became_active":"2023-07-06T21:40:53.192014+0000","last_became_peered":"2023-07-06T21:40:53.192014+0000","last_unstale":"2023-07-06T21:40:56.036427+0000","last_undegraded":"2023-07-06T21:40:56.036427+0000","last_fullsized":"2023-07-06T21:40:56.036427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:39:24.803257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482739+0000","last_change":"2023-07-06T21:40:53.681150+0000","last_active":"2023-07-06T21:40:56.482739+0000","last_peered":"2023-07-06T21:40:56.482739+0000","last_clean":"2023-07-06T21:40:56.482739+0000","last_became_active":"2023-07-06T21:40:53.680511+0000","last_became_peered":"2023-07-06T21:40:53.680511+0000","last_unstale":"2023-07-06T21:40:56.482739+0000","last_undegraded":"2023-07-06T21:40:56.482739+0000","last_fullsized":"2023-07-06T21:40:56.482739+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:48:25.388658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381686+0000","last_change":"2023-07-06T21:40:53.187483+0000","last_active":"2023-07-06T21:40:55.381686+0000","last_peered":"2023-07-06T21:40:55.381686+0000","last_clean":"2023-07-06T21:40:55.381686+0000","last_became_active":"2023-07-06T21:40:53.187095+0000","last_became_peered":"2023-07-06T21:40:53.187095+0000","last_unstale":"2023-07-06T21:40:55.381686+0000","last_undegraded":"2023-07-06T21:40:55.381686+0000","last_fullsized":"2023-07-06T21:40:55.381686+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:45:15.764685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684219+0000","last_change":"2023-07-06T21:40:53.684178+0000","last_active":"2023-07-06T21:40:55.684219+0000","last_peered":"2023-07-06T21:40:55.684219+0000","last_clean":"2023-07-06T21:40:55.684219+0000","last_became_active":"2023-07-06T21:40:53.683883+0000","last_became_peered":"2023-07-06T21:40:53.683883+0000","last_unstale":"2023-07-06T21:40:55.684219+0000","last_undegraded":"2023-07-06T21:40:55.684219+0000","last_fullsized":"2023-07-06T21:40:55.684219+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:19:34.782540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482526+0000","last_change":"2023-07-06T21:40:53.182219+0000","last_active":"2023-07-06T21:40:56.482526+0000","last_peered":"2023-07-06T21:40:56.482526+0000","last_clean":"2023-07-06T21:40:56.482526+0000","last_became_active":"2023-07-06T21:40:53.181951+0000","last_became_peered":"2023-07-06T21:40:53.181951+0000","last_unstale":"2023-07-06T21:40:56.482526+0000","last_undegraded":"2023-07-06T21:40:56.482526+0000","last_fullsized":"2023-07-06T21:40:56.482526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:32:23.388862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664338+0000","last_change":"2023-07-06T21:40:53.672834+0000","last_active":"2023-07-06T21:40:54.664338+0000","last_peered":"2023-07-06T21:40:54.664338+0000","last_clean":"2023-07-06T21:40:54.664338+0000","last_became_active":"2023-07-06T21:40:53.672525+0000","last_became_peered":"2023-07-06T21:40:53.672525+0000","last_unstale":"2023-07-06T21:40:54.664338+0000","last_undegraded":"2023-07-06T21:40:54.664338+0000","last_fullsized":"2023-07-06T21:40:54.664338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:13:27.700081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684335+0000","last_change":"2023-07-06T21:40:53.188952+0000","last_active":"2023-07-06T21:40:55.684335+0000","last_peered":"2023-07-06T21:40:55.684335+0000","last_clean":"2023-07-06T21:40:55.684335+0000","last_became_active":"2023-07-06T21:40:53.188701+0000","last_became_peered":"2023-07-06T21:40:53.188701+0000","last_unstale":"2023-07-06T21:40:55.684335+0000","last_undegraded":"2023-07-06T21:40:55.684335+0000","last_fullsized":"2023-07-06T21:40:55.684335+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:09:04.945994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036797+0000","last_change":"2023-07-06T21:40:53.693391+0000","last_active":"2023-07-06T21:40:56.036797+0000","last_peered":"2023-07-06T21:40:56.036797+0000","last_clean":"2023-07-06T21:40:56.036797+0000","last_became_active":"2023-07-06T21:40:53.692733+0000","last_became_peered":"2023-07-06T21:40:53.692733+0000","last_unstale":"2023-07-06T21:40:56.036797+0000","last_undegraded":"2023-07-06T21:40:56.036797+0000","last_fullsized":"2023-07-06T21:40:56.036797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:03:53.052919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668563+0000","last_change":"2023-07-06T21:40:53.187592+0000","last_active":"2023-07-06T21:40:54.668563+0000","last_peered":"2023-07-06T21:40:54.668563+0000","last_clean":"2023-07-06T21:40:54.668563+0000","last_became_active":"2023-07-06T21:40:53.187241+0000","last_became_peered":"2023-07-06T21:40:53.187241+0000","last_unstale":"2023-07-06T21:40:54.668563+0000","last_undegraded":"2023-07-06T21:40:54.668563+0000","last_fullsized":"2023-07-06T21:40:54.668563+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:58:34.089052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482207+0000","last_change":"2023-07-06T21:40:53.689735+0000","last_active":"2023-07-06T21:40:56.482207+0000","last_peered":"2023-07-06T21:40:56.482207+0000","last_clean":"2023-07-06T21:40:56.482207+0000","last_became_active":"2023-07-06T21:40:53.689118+0000","last_became_peered":"2023-07-06T21:40:53.689118+0000","last_unstale":"2023-07-06T21:40:56.482207+0000","last_undegraded":"2023-07-06T21:40:56.482207+0000","last_fullsized":"2023-07-06T21:40:56.482207+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:43:54.902088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668401+0000","last_change":"2023-07-06T21:40:53.183990+0000","last_active":"2023-07-06T21:40:54.668401+0000","last_peered":"2023-07-06T21:40:54.668401+0000","last_clean":"2023-07-06T21:40:54.668401+0000","last_became_active":"2023-07-06T21:40:53.183694+0000","last_became_peered":"2023-07-06T21:40:53.183694+0000","last_unstale":"2023-07-06T21:40:54.668401+0000","last_undegraded":"2023-07-06T21:40:54.668401+0000","last_fullsized":"2023-07-06T21:40:54.668401+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:48:24.380571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483074+0000","last_change":"2023-07-06T21:40:53.680883+0000","last_active":"2023-07-06T21:40:56.483074+0000","last_peered":"2023-07-06T21:40:56.483074+0000","last_clean":"2023-07-06T21:40:56.483074+0000","last_became_active":"2023-07-06T21:40:53.680527+0000","last_became_peered":"2023-07-06T21:40:53.680527+0000","last_unstale":"2023-07-06T21:40:56.483074+0000","last_undegraded":"2023-07-06T21:40:56.483074+0000","last_fullsized":"2023-07-06T21:40:56.483074+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:58:35.097140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684171+0000","last_change":"2023-07-06T21:40:53.190453+0000","last_active":"2023-07-06T21:40:55.684171+0000","last_peered":"2023-07-06T21:40:55.684171+0000","last_clean":"2023-07-06T21:40:55.684171+0000","last_became_active":"2023-07-06T21:40:53.190201+0000","last_became_peered":"2023-07-06T21:40:53.190201+0000","last_unstale":"2023-07-06T21:40:55.684171+0000","last_undegraded":"2023-07-06T21:40:55.684171+0000","last_fullsized":"2023-07-06T21:40:55.684171+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:40:25.368063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482035+0000","last_change":"2023-07-06T21:40:53.690053+0000","last_active":"2023-07-06T21:40:56.482035+0000","last_peered":"2023-07-06T21:40:56.482035+0000","last_clean":"2023-07-06T21:40:56.482035+0000","last_became_active":"2023-07-06T21:40:53.689804+0000","last_became_peered":"2023-07-06T21:40:53.689804+0000","last_unstale":"2023-07-06T21:40:56.482035+0000","last_undegraded":"2023-07-06T21:40:56.482035+0000","last_fullsized":"2023-07-06T21:40:56.482035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:59:08.486027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684240+0000","last_change":"2023-07-06T21:40:53.187472+0000","last_active":"2023-07-06T21:40:55.684240+0000","last_peered":"2023-07-06T21:40:55.684240+0000","last_clean":"2023-07-06T21:40:55.684240+0000","last_became_active":"2023-07-06T21:40:53.187247+0000","last_became_peered":"2023-07-06T21:40:53.187247+0000","last_unstale":"2023-07-06T21:40:55.684240+0000","last_undegraded":"2023-07-06T21:40:55.684240+0000","last_fullsized":"2023-07-06T21:40:55.684240+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:01:32.938122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.683877+0000","last_change":"2023-07-06T21:40:53.682812+0000","last_active":"2023-07-06T21:40:55.683877+0000","last_peered":"2023-07-06T21:40:55.683877+0000","last_clean":"2023-07-06T21:40:55.683877+0000","last_became_active":"2023-07-06T21:40:53.682518+0000","last_became_peered":"2023-07-06T21:40:53.682518+0000","last_unstale":"2023-07-06T21:40:55.683877+0000","last_undegraded":"2023-07-06T21:40:55.683877+0000","last_fullsized":"2023-07-06T21:40:55.683877+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:03:28.083865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381580+0000","last_change":"2023-07-06T21:40:53.186493+0000","last_active":"2023-07-06T21:40:55.381580+0000","last_peered":"2023-07-06T21:40:55.381580+0000","last_clean":"2023-07-06T21:40:55.381580+0000","last_became_active":"2023-07-06T21:40:53.186188+0000","last_became_peered":"2023-07-06T21:40:53.186188+0000","last_unstale":"2023-07-06T21:40:55.381580+0000","last_undegraded":"2023-07-06T21:40:55.381580+0000","last_fullsized":"2023-07-06T21:40:55.381580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:39:13.929085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036670+0000","last_change":"2023-07-06T21:40:53.676345+0000","last_active":"2023-07-06T21:40:56.036670+0000","last_peered":"2023-07-06T21:40:56.036670+0000","last_clean":"2023-07-06T21:40:56.036670+0000","last_became_active":"2023-07-06T21:40:53.676040+0000","last_became_peered":"2023-07-06T21:40:53.676040+0000","last_unstale":"2023-07-06T21:40:56.036670+0000","last_undegraded":"2023-07-06T21:40:56.036670+0000","last_fullsized":"2023-07-06T21:40:56.036670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:17:31.464007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667398+0000","last_change":"2023-07-06T21:40:53.186727+0000","last_active":"2023-07-06T21:40:54.667398+0000","last_peered":"2023-07-06T21:40:54.667398+0000","last_clean":"2023-07-06T21:40:54.667398+0000","last_became_active":"2023-07-06T21:40:53.186335+0000","last_became_peered":"2023-07-06T21:40:53.186335+0000","last_unstale":"2023-07-06T21:40:54.667398+0000","last_undegraded":"2023-07-06T21:40:54.667398+0000","last_fullsized":"2023-07-06T21:40:54.667398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:58.270164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684008+0000","last_change":"2023-07-06T21:40:53.682830+0000","last_active":"2023-07-06T21:40:55.684008+0000","last_peered":"2023-07-06T21:40:55.684008+0000","last_clean":"2023-07-06T21:40:55.684008+0000","last_became_active":"2023-07-06T21:40:53.682587+0000","last_became_peered":"2023-07-06T21:40:53.682587+0000","last_unstale":"2023-07-06T21:40:55.684008+0000","last_undegraded":"2023-07-06T21:40:55.684008+0000","last_fullsized":"2023-07-06T21:40:55.684008+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:52:48.741999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036794+0000","last_change":"2023-07-06T21:40:53.192564+0000","last_active":"2023-07-06T21:40:56.036794+0000","last_peered":"2023-07-06T21:40:56.036794+0000","last_clean":"2023-07-06T21:40:56.036794+0000","last_became_active":"2023-07-06T21:40:53.192281+0000","last_became_peered":"2023-07-06T21:40:53.192281+0000","last_unstale":"2023-07-06T21:40:56.036794+0000","last_undegraded":"2023-07-06T21:40:56.036794+0000","last_fullsized":"2023-07-06T21:40:56.036794+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:10:57.566657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684330+0000","last_change":"2023-07-06T21:40:53.688292+0000","last_active":"2023-07-06T21:40:55.684330+0000","last_peered":"2023-07-06T21:40:55.684330+0000","last_clean":"2023-07-06T21:40:55.684330+0000","last_became_active":"2023-07-06T21:40:53.688047+0000","last_became_peered":"2023-07-06T21:40:53.688047+0000","last_unstale":"2023-07-06T21:40:55.684330+0000","last_undegraded":"2023-07-06T21:40:55.684330+0000","last_fullsized":"2023-07-06T21:40:55.684330+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:41:19.252901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668649+0000","last_change":"2023-07-06T21:40:53.178836+0000","last_active":"2023-07-06T21:40:54.668649+0000","last_peered":"2023-07-06T21:40:54.668649+0000","last_clean":"2023-07-06T21:40:54.668649+0000","last_became_active":"2023-07-06T21:40:53.178562+0000","last_became_peered":"2023-07-06T21:40:53.178562+0000","last_unstale":"2023-07-06T21:40:54.668649+0000","last_undegraded":"2023-07-06T21:40:54.668649+0000","last_fullsized":"2023-07-06T21:40:54.668649+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:52:02.866459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668186+0000","last_change":"2023-07-06T21:40:53.686558+0000","last_active":"2023-07-06T21:40:54.668186+0000","last_peered":"2023-07-06T21:40:54.668186+0000","last_clean":"2023-07-06T21:40:54.668186+0000","last_became_active":"2023-07-06T21:40:53.686216+0000","last_became_peered":"2023-07-06T21:40:53.686216+0000","last_unstale":"2023-07-06T21:40:54.668186+0000","last_undegraded":"2023-07-06T21:40:54.668186+0000","last_fullsized":"2023-07-06T21:40:54.668186+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:35:46.789829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483153+0000","last_change":"2023-07-06T21:40:53.183012+0000","last_active":"2023-07-06T21:40:56.483153+0000","last_peered":"2023-07-06T21:40:56.483153+0000","last_clean":"2023-07-06T21:40:56.483153+0000","last_became_active":"2023-07-06T21:40:53.182747+0000","last_became_peered":"2023-07-06T21:40:53.182747+0000","last_unstale":"2023-07-06T21:40:56.483153+0000","last_undegraded":"2023-07-06T21:40:56.483153+0000","last_fullsized":"2023-07-06T21:40:56.483153+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:32:48.463180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.483131+0000","last_change":"2023-07-06T21:40:53.678992+0000","last_active":"2023-07-06T21:40:56.483131+0000","last_peered":"2023-07-06T21:40:56.483131+0000","last_clean":"2023-07-06T21:40:56.483131+0000","last_became_active":"2023-07-06T21:40:53.678689+0000","last_became_peered":"2023-07-06T21:40:53.678689+0000","last_unstale":"2023-07-06T21:40:56.483131+0000","last_undegraded":"2023-07-06T21:40:56.483131+0000","last_fullsized":"2023-07-06T21:40:56.483131+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:52:03.874547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664090+0000","last_change":"2023-07-06T21:40:53.189862+0000","last_active":"2023-07-06T21:40:54.664090+0000","last_peered":"2023-07-06T21:40:54.664090+0000","last_clean":"2023-07-06T21:40:54.664090+0000","last_became_active":"2023-07-06T21:40:53.189524+0000","last_became_peered":"2023-07-06T21:40:53.189524+0000","last_unstale":"2023-07-06T21:40:54.664090+0000","last_undegraded":"2023-07-06T21:40:54.664090+0000","last_fullsized":"2023-07-06T21:40:54.664090+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:31:40.687007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664236+0000","last_change":"2023-07-06T21:40:53.678022+0000","last_active":"2023-07-06T21:40:54.664236+0000","last_peered":"2023-07-06T21:40:54.664236+0000","last_clean":"2023-07-06T21:40:54.664236+0000","last_became_active":"2023-07-06T21:40:53.677774+0000","last_became_peered":"2023-07-06T21:40:53.677774+0000","last_unstale":"2023-07-06T21:40:54.664236+0000","last_undegraded":"2023-07-06T21:40:54.664236+0000","last_fullsized":"2023-07-06T21:40:54.664236+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:45:50.000441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381759+0000","last_change":"2023-07-06T21:40:53.185678+0000","last_active":"2023-07-06T21:40:55.381759+0000","last_peered":"2023-07-06T21:40:55.381759+0000","last_clean":"2023-07-06T21:40:55.381759+0000","last_became_active":"2023-07-06T21:40:53.185383+0000","last_became_peered":"2023-07-06T21:40:53.185383+0000","last_unstale":"2023-07-06T21:40:55.381759+0000","last_undegraded":"2023-07-06T21:40:55.381759+0000","last_fullsized":"2023-07-06T21:40:55.381759+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:17:52.737294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036745+0000","last_change":"2023-07-06T21:40:53.696932+0000","last_active":"2023-07-06T21:40:56.036745+0000","last_peered":"2023-07-06T21:40:56.036745+0000","last_clean":"2023-07-06T21:40:56.036745+0000","last_became_active":"2023-07-06T21:40:53.689997+0000","last_became_peered":"2023-07-06T21:40:53.689997+0000","last_unstale":"2023-07-06T21:40:56.036745+0000","last_undegraded":"2023-07-06T21:40:56.036745+0000","last_fullsized":"2023-07-06T21:40:56.036745+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:41:19.252901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663990+0000","last_change":"2023-07-06T21:40:53.179436+0000","last_active":"2023-07-06T21:40:54.663990+0000","last_peered":"2023-07-06T21:40:54.663990+0000","last_clean":"2023-07-06T21:40:54.663990+0000","last_became_active":"2023-07-06T21:40:53.179178+0000","last_became_peered":"2023-07-06T21:40:53.179178+0000","last_unstale":"2023-07-06T21:40:54.663990+0000","last_undegraded":"2023-07-06T21:40:54.663990+0000","last_fullsized":"2023-07-06T21:40:54.663990+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:50:28.361917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.481979+0000","last_change":"2023-07-06T21:40:53.689719+0000","last_active":"2023-07-06T21:40:56.481979+0000","last_peered":"2023-07-06T21:40:56.481979+0000","last_clean":"2023-07-06T21:40:56.481979+0000","last_became_active":"2023-07-06T21:40:53.689430+0000","last_became_peered":"2023-07-06T21:40:53.689430+0000","last_unstale":"2023-07-06T21:40:56.481979+0000","last_undegraded":"2023-07-06T21:40:56.481979+0000","last_fullsized":"2023-07-06T21:40:56.481979+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:35:04.612510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663962+0000","last_change":"2023-07-06T21:40:53.176020+0000","last_active":"2023-07-06T21:40:54.663962+0000","last_peered":"2023-07-06T21:40:54.663962+0000","last_clean":"2023-07-06T21:40:54.663962+0000","last_became_active":"2023-07-06T21:40:53.175748+0000","last_became_peered":"2023-07-06T21:40:53.175748+0000","last_unstale":"2023-07-06T21:40:54.663962+0000","last_undegraded":"2023-07-06T21:40:54.663962+0000","last_fullsized":"2023-07-06T21:40:54.663962+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:15:58.270164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667500+0000","last_change":"2023-07-06T21:40:53.692043+0000","last_active":"2023-07-06T21:40:54.667500+0000","last_peered":"2023-07-06T21:40:54.667500+0000","last_clean":"2023-07-06T21:40:54.667500+0000","last_became_active":"2023-07-06T21:40:53.691745+0000","last_became_peered":"2023-07-06T21:40:53.691745+0000","last_unstale":"2023-07-06T21:40:54.667500+0000","last_undegraded":"2023-07-06T21:40:54.667500+0000","last_fullsized":"2023-07-06T21:40:54.667500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:09:41.600615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036555+0000","last_change":"2023-07-06T21:40:53.189895+0000","last_active":"2023-07-06T21:40:56.036555+0000","last_peered":"2023-07-06T21:40:56.036555+0000","last_clean":"2023-07-06T21:40:56.036555+0000","last_became_active":"2023-07-06T21:40:53.189488+0000","last_became_peered":"2023-07-06T21:40:53.189488+0000","last_unstale":"2023-07-06T21:40:56.036555+0000","last_undegraded":"2023-07-06T21:40:56.036555+0000","last_fullsized":"2023-07-06T21:40:56.036555+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:34:41.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684130+0000","last_change":"2023-07-06T21:40:53.684323+0000","last_active":"2023-07-06T21:40:55.684130+0000","last_peered":"2023-07-06T21:40:55.684130+0000","last_clean":"2023-07-06T21:40:55.684130+0000","last_became_active":"2023-07-06T21:40:53.684041+0000","last_became_peered":"2023-07-06T21:40:53.684041+0000","last_unstale":"2023-07-06T21:40:55.684130+0000","last_undegraded":"2023-07-06T21:40:55.684130+0000","last_fullsized":"2023-07-06T21:40:55.684130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:47:11.828608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663854+0000","last_change":"2023-07-06T21:40:53.181447+0000","last_active":"2023-07-06T21:40:54.663854+0000","last_peered":"2023-07-06T21:40:54.663854+0000","last_clean":"2023-07-06T21:40:54.663854+0000","last_became_active":"2023-07-06T21:40:53.181209+0000","last_became_peered":"2023-07-06T21:40:53.181209+0000","last_unstale":"2023-07-06T21:40:54.663854+0000","last_undegraded":"2023-07-06T21:40:54.663854+0000","last_fullsized":"2023-07-06T21:40:54.663854+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:00:08.281271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.481867+0000","last_change":"2023-07-06T21:40:53.681810+0000","last_active":"2023-07-06T21:40:56.481867+0000","last_peered":"2023-07-06T21:40:56.481867+0000","last_clean":"2023-07-06T21:40:56.481867+0000","last_became_active":"2023-07-06T21:40:53.681490+0000","last_became_peered":"2023-07-06T21:40:53.681490+0000","last_unstale":"2023-07-06T21:40:56.481867+0000","last_undegraded":"2023-07-06T21:40:56.481867+0000","last_fullsized":"2023-07-06T21:40:56.481867+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:36:09.578251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482064+0000","last_change":"2023-07-06T21:40:53.185048+0000","last_active":"2023-07-06T21:40:56.482064+0000","last_peered":"2023-07-06T21:40:56.482064+0000","last_clean":"2023-07-06T21:40:56.482064+0000","last_became_active":"2023-07-06T21:40:53.184786+0000","last_became_peered":"2023-07-06T21:40:53.184786+0000","last_unstale":"2023-07-06T21:40:56.482064+0000","last_undegraded":"2023-07-06T21:40:56.482064+0000","last_fullsized":"2023-07-06T21:40:56.482064+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:47:57.673943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381881+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:40:55.381881+0000","last_peered":"2023-07-06T21:40:55.381881+0000","last_clean":"2023-07-06T21:40:55.381881+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:40:55.381881+0000","last_undegraded":"2023-07-06T21:40:55.381881+0000","last_fullsized":"2023-07-06T21:40:55.381881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:39:45.590139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.684196+0000","last_change":"2023-07-06T21:40:53.677480+0000","last_active":"2023-07-06T21:40:55.684196+0000","last_peered":"2023-07-06T21:40:55.684196+0000","last_clean":"2023-07-06T21:40:55.684196+0000","last_became_active":"2023-07-06T21:40:53.677213+0000","last_became_peered":"2023-07-06T21:40:53.677213+0000","last_unstale":"2023-07-06T21:40:55.684196+0000","last_undegraded":"2023-07-06T21:40:55.684196+0000","last_fullsized":"2023-07-06T21:40:55.684196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:45:16.772773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482185+0000","last_change":"2023-07-06T21:40:53.183577+0000","last_active":"2023-07-06T21:40:56.482185+0000","last_peered":"2023-07-06T21:40:56.482185+0000","last_clean":"2023-07-06T21:40:56.482185+0000","last_became_active":"2023-07-06T21:40:53.183328+0000","last_became_peered":"2023-07-06T21:40:53.183328+0000","last_unstale":"2023-07-06T21:40:56.482185+0000","last_undegraded":"2023-07-06T21:40:56.482185+0000","last_fullsized":"2023-07-06T21:40:56.482185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:21:30.938062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668415+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:40:54.668415+0000","last_peered":"2023-07-06T21:40:54.668415+0000","last_clean":"2023-07-06T21:40:54.668415+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:40:54.668415+0000","last_undegraded":"2023-07-06T21:40:54.668415+0000","last_fullsized":"2023-07-06T21:40:54.668415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:21:41.048106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036908+0000","last_change":"2023-07-06T21:40:53.693490+0000","last_active":"2023-07-06T21:40:56.036908+0000","last_peered":"2023-07-06T21:40:56.036908+0000","last_clean":"2023-07-06T21:40:56.036908+0000","last_became_active":"2023-07-06T21:40:53.693146+0000","last_became_peered":"2023-07-06T21:40:53.693146+0000","last_unstale":"2023-07-06T21:40:56.036908+0000","last_undegraded":"2023-07-06T21:40:56.036908+0000","last_fullsized":"2023-07-06T21:40:56.036908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:37.360689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.664142+0000","last_change":"2023-07-06T21:40:53.175426+0000","last_active":"2023-07-06T21:40:54.664142+0000","last_peered":"2023-07-06T21:40:54.664142+0000","last_clean":"2023-07-06T21:40:54.664142+0000","last_became_active":"2023-07-06T21:40:53.175023+0000","last_became_peered":"2023-07-06T21:40:53.175023+0000","last_unstale":"2023-07-06T21:40:54.664142+0000","last_undegraded":"2023-07-06T21:40:54.664142+0000","last_fullsized":"2023-07-06T21:40:54.664142+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:06:24.996298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.663869+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:40:54.663869+0000","last_peered":"2023-07-06T21:40:54.663869+0000","last_clean":"2023-07-06T21:40:54.663869+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:40:54.663869+0000","last_undegraded":"2023-07-06T21:40:54.663869+0000","last_fullsized":"2023-07-06T21:40:54.663869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:16:23.533601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668503+0000","last_change":"2023-07-06T21:40:53.689180+0000","last_active":"2023-07-06T21:40:54.668503+0000","last_peered":"2023-07-06T21:40:54.668503+0000","last_clean":"2023-07-06T21:40:54.668503+0000","last_became_active":"2023-07-06T21:40:53.688790+0000","last_became_peered":"2023-07-06T21:40:53.688790+0000","last_unstale":"2023-07-06T21:40:54.668503+0000","last_undegraded":"2023-07-06T21:40:54.668503+0000","last_fullsized":"2023-07-06T21:40:54.668503+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:02:53.877968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036523+0000","last_change":"2023-07-06T21:40:53.188502+0000","last_active":"2023-07-06T21:40:56.036523+0000","last_peered":"2023-07-06T21:40:56.036523+0000","last_clean":"2023-07-06T21:40:56.036523+0000","last_became_active":"2023-07-06T21:40:53.188181+0000","last_became_peered":"2023-07-06T21:40:53.188181+0000","last_unstale":"2023-07-06T21:40:56.036523+0000","last_undegraded":"2023-07-06T21:40:56.036523+0000","last_fullsized":"2023-07-06T21:40:56.036523+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:09:40.592527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668402+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:40:54.668402+0000","last_peered":"2023-07-06T21:40:54.668402+0000","last_clean":"2023-07-06T21:40:54.668402+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:40:54.668402+0000","last_undegraded":"2023-07-06T21:40:54.668402+0000","last_fullsized":"2023-07-06T21:40:54.668402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:47.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036843+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:40:56.036843+0000","last_peered":"2023-07-06T21:40:56.036843+0000","last_clean":"2023-07-06T21:40:56.036843+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:40:56.036843+0000","last_undegraded":"2023-07-06T21:40:56.036843+0000","last_fullsized":"2023-07-06T21:40:56.036843+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:30:43.581561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381776+0000","last_change":"2023-07-06T21:40:53.692690+0000","last_active":"2023-07-06T21:40:55.381776+0000","last_peered":"2023-07-06T21:40:55.381776+0000","last_clean":"2023-07-06T21:40:55.381776+0000","last_became_active":"2023-07-06T21:40:53.692398+0000","last_became_peered":"2023-07-06T21:40:53.692398+0000","last_unstale":"2023-07-06T21:40:55.381776+0000","last_undegraded":"2023-07-06T21:40:55.381776+0000","last_fullsized":"2023-07-06T21:40:55.381776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:19:34.782540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668650+0000","last_change":"2023-07-06T21:40:53.184527+0000","last_active":"2023-07-06T21:40:54.668650+0000","last_peered":"2023-07-06T21:40:54.668650+0000","last_clean":"2023-07-06T21:40:54.668650+0000","last_became_active":"2023-07-06T21:40:53.184194+0000","last_became_peered":"2023-07-06T21:40:53.184194+0000","last_unstale":"2023-07-06T21:40:54.668650+0000","last_undegraded":"2023-07-06T21:40:54.668650+0000","last_fullsized":"2023-07-06T21:40:54.668650+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:25:27.793540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036974+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:40:56.036974+0000","last_peered":"2023-07-06T21:40:56.036974+0000","last_clean":"2023-07-06T21:40:56.036974+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:40:56.036974+0000","last_undegraded":"2023-07-06T21:40:56.036974+0000","last_fullsized":"2023-07-06T21:40:56.036974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:46:51.725897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381516+0000","last_change":"2023-07-06T21:40:53.685517+0000","last_active":"2023-07-06T21:40:55.381516+0000","last_peered":"2023-07-06T21:40:55.381516+0000","last_clean":"2023-07-06T21:40:55.381516+0000","last_became_active":"2023-07-06T21:40:53.685054+0000","last_became_peered":"2023-07-06T21:40:53.685054+0000","last_unstale":"2023-07-06T21:40:55.381516+0000","last_undegraded":"2023-07-06T21:40:55.381516+0000","last_fullsized":"2023-07-06T21:40:55.381516+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:34:05.003871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.667782+0000","last_change":"2023-07-06T21:40:53.183316+0000","last_active":"2023-07-06T21:40:54.667782+0000","last_peered":"2023-07-06T21:40:54.667782+0000","last_clean":"2023-07-06T21:40:54.667782+0000","last_became_active":"2023-07-06T21:40:53.182240+0000","last_became_peered":"2023-07-06T21:40:53.182240+0000","last_unstale":"2023-07-06T21:40:54.667782+0000","last_undegraded":"2023-07-06T21:40:54.667782+0000","last_fullsized":"2023-07-06T21:40:54.667782+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:27:16.473263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036958+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:40:56.036958+0000","last_peered":"2023-07-06T21:40:56.036958+0000","last_clean":"2023-07-06T21:40:56.036958+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:40:56.036958+0000","last_undegraded":"2023-07-06T21:40:56.036958+0000","last_fullsized":"2023-07-06T21:40:56.036958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:47.248044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482889+0000","last_change":"2023-07-06T21:40:53.677627+0000","last_active":"2023-07-06T21:40:56.482889+0000","last_peered":"2023-07-06T21:40:56.482889+0000","last_clean":"2023-07-06T21:40:56.482889+0000","last_became_active":"2023-07-06T21:40:53.677341+0000","last_became_peered":"2023-07-06T21:40:53.677341+0000","last_unstale":"2023-07-06T21:40:56.482889+0000","last_undegraded":"2023-07-06T21:40:56.482889+0000","last_fullsized":"2023-07-06T21:40:56.482889+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:23:14.710868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.482609+0000","last_change":"2023-07-06T21:40:53.189329+0000","last_active":"2023-07-06T21:40:56.482609+0000","last_peered":"2023-07-06T21:40:56.482609+0000","last_clean":"2023-07-06T21:40:56.482609+0000","last_became_active":"2023-07-06T21:40:53.189041+0000","last_became_peered":"2023-07-06T21:40:53.189041+0000","last_unstale":"2023-07-06T21:40:56.482609+0000","last_undegraded":"2023-07-06T21:40:56.482609+0000","last_fullsized":"2023-07-06T21:40:56.482609+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:40:19.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.381700+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:40:55.381700+0000","last_peered":"2023-07-06T21:40:55.381700+0000","last_clean":"2023-07-06T21:40:55.381700+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:40:55.381700+0000","last_undegraded":"2023-07-06T21:40:55.381700+0000","last_fullsized":"2023-07-06T21:40:55.381700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:52.713389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:56.036857+0000","last_change":"2023-07-06T21:40:53.686880+0000","last_active":"2023-07-06T21:40:56.036857+0000","last_peered":"2023-07-06T21:40:56.036857+0000","last_clean":"2023-07-06T21:40:56.036857+0000","last_became_active":"2023-07-06T21:40:53.686581+0000","last_became_peered":"2023-07-06T21:40:53.686581+0000","last_unstale":"2023-07-06T21:40:56.036857+0000","last_undegraded":"2023-07-06T21:40:56.036857+0000","last_fullsized":"2023-07-06T21:40:56.036857+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:53.160205+0000","last_clean_scrub_stamp":"2023-07-06T21:40:53.160205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:29:22.655067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:55.683887+0000","last_change":"2023-07-06T21:40:53.184306+0000","last_active":"2023-07-06T21:40:55.683887+0000","last_peered":"2023-07-06T21:40:55.683887+0000","last_clean":"2023-07-06T21:40:55.683887+0000","last_became_active":"2023-07-06T21:40:53.184027+0000","last_became_peered":"2023-07-06T21:40:53.184027+0000","last_unstale":"2023-07-06T21:40:55.683887+0000","last_undegraded":"2023-07-06T21:40:55.683887+0000","last_fullsized":"2023-07-06T21:40:55.683887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:40:52.152117+0000","last_clean_scrub_stamp":"2023-07-06T21:40:52.152117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:15:06.526265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":39,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-07-06T21:40:54.668628+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:40:54.668628+0000","last_peered":"2023-07-06T21:40:54.668628+0000","last_clean":"2023-07-06T21:40:54.668628+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:40:54.668628+0000","last_undegraded":"2023-07-06T21:40:54.668628+0000","last_fullsized":"2023-07-06T21:40:54.668628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:43.505095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":1008,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":1032192,"data_stored":767390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.28,"5min":1.28,"15min":1.28},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.81100000000000005}]},{"osd":4,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.86599999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.379,"5min":1.379,"15min":1.379},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.97599999999999998}]},{"osd":6,"last update":"Thu Jul 6 21:40:16 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9279999999999999,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.996}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":3.569,"5min":3.569,"15min":3.569},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.69499999999999995}]},{"osd":1,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":3.6779999999999999,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":3.7970000000000002,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":3.6400000000000001,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.71999999999999997}]},{"osd":3,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.6200000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.875}]},{"osd":4,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80700000000000005}]},{"osd":5,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.86199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":5.5129999999999999,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":2.246,"5min":2.246,"15min":2.246},"last":0.442}]},{"osd":2,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.2709999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":11.913,"5min":11.913,"15min":11.913},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":22.053000000000001,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":9.1920000000000002,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":5.4790000000000001,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.622}]},{"osd":5,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.2949999999999999,"5min":2.2949999999999999,"15min":2.2949999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":3.2280000000000002,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.70699999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.4349999999999996,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.58199999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":1015808,"data_stored":764771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":2.7869999999999999,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.433}]},{"osd":2,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.3319999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.377},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.34300000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.234,"5min":2.234,"15min":2.234},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":3.5859999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":3.544,"5min":3.544,"15min":3.544},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.367,"5min":2.367,"15min":2.367},"last":0.63800000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":2.395,"5min":2.395,"15min":2.395},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.2629999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.6190000000000002,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":0.58099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":416,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":425984,"data_stored":175458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.502}]},{"osd":4,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.621}]},{"osd":5,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:40:12 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":438272,"data_stored":176986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.44800000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.55900000000000005}]},{"osd":2,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.59999999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.8640000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.83499999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":2.012,"5min":2.012,"15min":2.012},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.89100000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.86799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":1.871}]},{"osd":1,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.4650000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.82899999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":1.375},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":1.3100000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7030000000000001,"5min":1.7030000000000001,"15min":1.7030000000000001},"last":2.0550000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":1.431}]},{"osd":5,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.929}]},{"osd":6,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.819,"5min":1.819,"15min":1.819},"last":2.859},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.88}]},{"osd":7,"last update":"Thu Jul 6 21:40:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":2.1339999999999999},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":2.2410000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":420,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":430080,"data_stored":175500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.71899999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68300000000000005}]},{"osd":2,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69999999999999996}]},{"osd":3,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.81599999999999995}]},{"osd":4,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.83699999999999997}]},{"osd":6,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.80000000000000004}]},{"osd":7,"last update":"Thu Jul 6 21:40:11 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.96899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:40:58.811 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T21:40:58.811 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T21:40:59.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.043+0000 7f3f625ff700 1 Processor -- start 2023-07-06T21:40:59.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.043+0000 7f3f625ff700 1 -- start start 2023-07-06T21:40:59.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f625ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f5c0d0ea0 0x7f3f5c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f625ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f5c15fee0 0x7f3f5c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f625ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0ceef0 con 0x7f3f5c160ac0 2023-07-06T21:40:59.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f625ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cf450 con 0x7f3f5c0d0ea0 2023-07-06T21:40:59.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f625ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c160820 con 0x7f3f5c15fee0 2023-07-06T21:40:59.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f5b7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f5c160ac0 0x7f3f5c0cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48322/0 (socket says 172.21.15.84:48322) 2023-07-06T21:40:59.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f5b7fe700 1 -- 172.21.15.84:0/1271417456 learned_addr learned my addr 172.21.15.84:0/1271417456 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:40:59.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.044+0000 7f3f5bfff700 1 --2- 172.21.15.84:0/1271417456 >> v2:172.21.15.84:3300/0 conn(0x7f3f5c0d0ea0 0x7f3f5c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f60b9c700 1 --2- 172.21.15.84:0/1271417456 >> v2:172.21.15.107:3300/0 conn(0x7f3f5c15fee0 0x7f3f5c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5affd700 1 -- 172.21.15.84:0/1271417456 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800833154 0 0) 0x7f3f5c0ceef0 con 0x7f3f5c160ac0 2023-07-06T21:40:59.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5affd700 1 -- 172.21.15.84:0/1271417456 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f440035b0 con 0x7f3f5c160ac0 2023-07-06T21:40:59.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5bfff700 1 -- 172.21.15.84:0/1271417456 >> v2:172.21.15.107:3300/0 conn(0x7f3f5c15fee0 msgr2=0x7f3f5c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5bfff700 1 --2- 172.21.15.84:0/1271417456 >> v2:172.21.15.107:3300/0 conn(0x7f3f5c15fee0 0x7f3f5c1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5bfff700 1 -- 172.21.15.84:0/1271417456 >> v1:172.21.15.84:6789/0 conn(0x7f3f5c160ac0 legacy=0x7f3f5c0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5bfff700 1 -- 172.21.15.84:0/1271417456 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0cf120 con 0x7f3f5c0d0ea0 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5bfff700 1 --2- 172.21.15.84:0/1271417456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c0d0ea0 0x7f3f5c0d1280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0c9d00 tx=0x7f3f4c009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5265f314fcdd5e30 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5affd700 1 -- 172.21.15.84:0/1271417456 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c005e30 con 0x7f3f5c0d0ea0 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5affd700 1 -- 172.21.15.84:0/1271417456 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c006030 con 0x7f3f5c0d0ea0 2023-07-06T21:40:59.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.045+0000 7f3f5affd700 1 -- 172.21.15.84:0/1271417456 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0063a0 con 0x7f3f5c0d0ea0 2023-07-06T21:40:59.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 -- 172.21.15.84:0/1271417456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c0d0ea0 msgr2=0x7f3f5c0d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 --2- 172.21.15.84:0/1271417456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c0d0ea0 0x7f3f5c0d1280 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c0c9d00 tx=0x7f3f4c009b00 comp rx=0 tx=0).stop 2023-07-06T21:40:59.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 -- 172.21.15.84:0/1271417456 shutdown_connections 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 --2- 172.21.15.84:0/1271417456 >> v2:172.21.15.107:3300/0 conn(0x7f3f5c15fee0 0x7f3f5c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 --2- 172.21.15.84:0/1271417456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c0d0ea0 0x7f3f5c0d1280 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 -- 172.21.15.84:0/1271417456 >> 172.21.15.84:0/1271417456 conn(0x7f3f5c0715c0 msgr2=0x7f3f5c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 -- 172.21.15.84:0/1271417456 shutdown_connections 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.046+0000 7f3f625ff700 1 -- 172.21.15.84:0/1271417456 wait complete. 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.048+0000 7f3f625ff700 1 Processor -- start 2023-07-06T21:40:59.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.049+0000 7f3f625ff700 1 -- start start 2023-07-06T21:40:59.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f625ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 0x7f3f5c0c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f625ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f5c160ac0 0x7f3f5c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f625ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f5c0cce10 0x7f3f5c153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f625ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cc4b0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f625ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cb830 con 0x7f3f5c160ac0 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f625ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0cbb10 con 0x7f3f5c0cce10 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 0x7f3f5c0c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 0x7f3f5c0c8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53264/0 (socket says 172.21.15.84:53264) 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 -- 172.21.15.84:0/324677303 learned_addr learned my addr 172.21.15.84:0/324677303 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5b7fe700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f5c160ac0 0x7f3f5c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f60b9c700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f5c0cce10 0x7f3f5c153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 -- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f5c0cce10 msgr2=0x7f3f5c153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f5c0cce10 0x7f3f5c153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 -- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f5c160ac0 msgr2=0x7f3f5c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.050+0000 7f3f5bfff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f5c160ac0 0x7f3f5c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f5bfff700 1 -- 172.21.15.84:0/324677303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009430 con 0x7f3f5c15fee0 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f5b7fe700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f5c160ac0 0x7f3f5c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f5bfff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 0x7f3f5c0c8670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009b40 tx=0x7f3f4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:59.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c004450 con 0x7f3f5c15fee0 2023-07-06T21:40:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c0045d0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0cbdf0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0038e0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.051+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c153f50 con 0x7f3f5c15fee0 2023-07-06T21:40:59.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.052+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0cc5f0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.053+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f4c022070 con 0x7f3f5c15fee0 2023-07-06T21:40:59.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.054+0000 7f3f58ff9700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4806ace0 0x7f3f4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.055+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f4c08ad40 con 0x7f3f5c15fee0 2023-07-06T21:40:59.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.055+0000 7f3f5b7fe700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4806ace0 0x7f3f4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.055+0000 7f3f5b7fe700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4806ace0 0x7f3f4806d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c150c50 tx=0x7f3f50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:59.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.059+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f5c0cc5f0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.333+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3f5c0cc5f0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.339+0000 7f3f58ff9700 1 -- 172.21.15.84:0/324677303 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3f5c0cc5f0 con 0x7f3f5c15fee0 2023-07-06T21:40:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4806ace0 msgr2=0x7f3f4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4806ace0 0x7f3f4806d190 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c150c50 tx=0x7f3f50009040 comp rx=0 tx=0).stop 2023-07-06T21:40:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 msgr2=0x7f3f5c0c8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 0x7f3f5c0c8670 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c009b40 tx=0x7f3f4c004750 comp rx=0 tx=0).stop 2023-07-06T21:40:59.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 shutdown_connections 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4806ace0 0x7f3f4806d190 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f5c0cce10 0x7f3f5c153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f5c15fee0 0x7f3f5c0c8670 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 --2- 172.21.15.84:0/324677303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f5c160ac0 0x7f3f5c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 >> 172.21.15.84:0/324677303 conn(0x7f3f5c0715c0 msgr2=0x7f3f5c15c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 shutdown_connections 2023-07-06T21:40:59.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.342+0000 7f3f625ff700 1 -- 172.21.15.84:0/324677303 wait complete. 2023-07-06T21:40:59.346 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:40:59.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:40:59.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.553+0000 7fb47fdff700 1 Processor -- start 2023-07-06T21:40:59.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.553+0000 7fb47fdff700 1 -- start start 2023-07-06T21:40:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47fdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4780b83c0 0x7fb4780cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47fdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4780cfc60 0x7fb4780d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47fdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780d0a80 con 0x7fb4780d3710 2023-07-06T21:40:59.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47fdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780b8790 con 0x7fb4780cfc60 2023-07-06T21:40:59.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47fdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4780b4e70 con 0x7fb4780b83c0 2023-07-06T21:40:59.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47e39c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4780cfc60 0x7fb4780d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47e39c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4780cfc60 0x7fb4780d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53268/0 (socket says 172.21.15.84:53268) 2023-07-06T21:40:59.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47e39c700 1 -- 172.21.15.84:0/3340763627 learned_addr learned my addr 172.21.15.84:0/3340763627 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:59.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.554+0000 7fb47db9b700 1 --2- 172.21.15.84:0/3340763627 >> v2:172.21.15.107:3300/0 conn(0x7fb4780b83c0 0x7fb4780cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47e39c700 1 -- 172.21.15.84:0/3340763627 >> v2:172.21.15.107:3300/0 conn(0x7fb4780b83c0 msgr2=0x7fb4780cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:40:59.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47e39c700 1 --2- 172.21.15.84:0/3340763627 >> v2:172.21.15.107:3300/0 conn(0x7fb4780b83c0 0x7fb4780cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47e39c700 1 -- 172.21.15.84:0/3340763627 >> v1:172.21.15.107:6789/0 conn(0x7fb4780d3710 legacy=0x7fb4780cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47e39c700 1 -- 172.21.15.84:0/3340763627 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780b5150 con 0x7fb4780cfc60 2023-07-06T21:40:59.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47e39c700 1 --2- 172.21.15.84:0/3340763627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780d0f10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4780cf520 tx=0x7fb47000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7c84aa11e25d35 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47cb99700 1 -- 172.21.15.84:0/3340763627 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000e070 con 0x7fb4780cfc60 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47cb99700 1 -- 172.21.15.84:0/3340763627 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47000b9c0 con 0x7fb4780cfc60 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.555+0000 7fb47cb99700 1 -- 172.21.15.84:0/3340763627 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47000bd70 con 0x7fb4780cfc60 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 -- 172.21.15.84:0/3340763627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 msgr2=0x7fb4780d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 --2- 172.21.15.84:0/3340763627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780d0f10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fb4780cf520 tx=0x7fb47000b330 comp rx=0 tx=0).stop 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 -- 172.21.15.84:0/3340763627 shutdown_connections 2023-07-06T21:40:59.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 --2- 172.21.15.84:0/3340763627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780d0f10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 --2- 172.21.15.84:0/3340763627 >> v2:172.21.15.107:3300/0 conn(0x7fb4780b83c0 0x7fb4780cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 -- 172.21.15.84:0/3340763627 >> 172.21.15.84:0/3340763627 conn(0x7fb47806b350 msgr2=0x7fb47806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:59.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 -- 172.21.15.84:0/3340763627 shutdown_connections 2023-07-06T21:40:59.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.556+0000 7fb47fdff700 1 -- 172.21.15.84:0/3340763627 wait complete. 2023-07-06T21:40:59.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.557+0000 7fb47fdff700 1 Processor -- start 2023-07-06T21:40:59.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.558+0000 7fb47fdff700 1 -- start start 2023-07-06T21:40:59.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.558+0000 7fb47fdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4780b83c0 0x7fb4781f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.558+0000 7fb47fdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47fdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4780d3710 0x7fb4780c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47fdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780c6980 con 0x7fb4780cfc60 2023-07-06T21:40:59.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47fdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478157960 con 0x7fb4780b83c0 2023-07-06T21:40:59.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47fdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ca9e0 con 0x7fb4780d3710 2023-07-06T21:40:59.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53274/0 (socket says 172.21.15.84:53274) 2023-07-06T21:40:59.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 -- 172.21.15.84:0/1004046433 learned_addr learned my addr 172.21.15.84:0/1004046433 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:40:59.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47e39c700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4780d3710 0x7fb4780c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47db9b700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4780b83c0 0x7fb4781f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 -- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4780d3710 msgr2=0x7fb4780c7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4780d3710 0x7fb4780c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 -- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4780b83c0 msgr2=0x7fb4781f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4780b83c0 0x7fb4781f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 -- 172.21.15.84:0/1004046433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb470009b30 con 0x7fb4780cfc60 2023-07-06T21:40:59.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47db9b700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4780b83c0 0x7fb4781f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.559+0000 7fb47d39a700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780c6fd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c4e0 tx=0x7fb46800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.560+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680052d0 con 0x7fb4780cfc60 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.560+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468005430 con 0x7fb4780cfc60 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.560+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4680038c0 con 0x7fb4780cfc60 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.560+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4780caca0 con 0x7fb4780cfc60 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.560+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780cb260 con 0x7fb4780cfc60 2023-07-06T21:40:59.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.561+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4780cd1c0 con 0x7fb4780cfc60 2023-07-06T21:40:59.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.562+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb468003a20 con 0x7fb4780cfc60 2023-07-06T21:40:59.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.563+0000 7fb46e7fc700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb45c06ace0 0x7fb45c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:40:59.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.564+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb4680889e0 con 0x7fb4780cfc60 2023-07-06T21:40:59.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.564+0000 7fb47db9b700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb45c06ace0 0x7fb45c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:40:59.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.564+0000 7fb47db9b700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb45c06ace0 0x7fb45c06d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb474009820 tx=0x7fb474008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:40:59.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.568+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb4680539f0 con 0x7fb4780cfc60 2023-07-06T21:40:59.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.811+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4780d1530 con 0x7fb4780cfc60 2023-07-06T21:40:59.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.811+0000 7fb46e7fc700 1 -- 172.21.15.84:0/1004046433 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4671 (secure 0 0 0) 0x7fb468057010 con 0x7fb4780cfc60 2023-07-06T21:40:59.813 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:40:59.814 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:40:56.959728+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:40:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.813+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb45c06ace0 msgr2=0x7fb45c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.813+0000 7fb47fdff700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb45c06ace0 0x7fb45c06d190 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb474009820 tx=0x7fb474008040 comp rx=0 tx=0).stop 2023-07-06T21:40:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.813+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 msgr2=0x7fb4780c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:40:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.813+0000 7fb47fdff700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780c6fd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb46800c4e0 tx=0x7fb46800c8a0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 shutdown_connections 2023-07-06T21:40:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb45c06ace0 0x7fb45c06d190 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4780d3710 0x7fb4780c7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4780b83c0 0x7fb4781f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 --2- 172.21.15.84:0/1004046433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4780cfc60 0x7fb4780c6fd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:40:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 >> 172.21.15.84:0/1004046433 conn(0x7fb47806b350 msgr2=0x7fb4780b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:40:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 shutdown_connections 2023-07-06T21:40:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:40:59.814+0000 7fb47fdff700 1 -- 172.21.15.84:0/1004046433 wait complete. 2023-07-06T21:40:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 15 2023-07-06T21:41:00.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T21:40:53.657594+0000', 'modified': '2023-07-06T21:40:56.959728+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5319}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5319': {'gid': 5319, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.84:6837/687288152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6835', 'nonce': 687288152}, {'type': 'v1', 'addr': '172.21.15.84:6837', 'nonce': 687288152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T21:41:00.237 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T21:41:00.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-07-06T21:41:00.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.431+0000 7fc981dff700 1 Processor -- start 2023-07-06T21:41:00.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.431+0000 7fc981dff700 1 -- start start 2023-07-06T21:41:00.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.432+0000 7fc981dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc97c1608f0 0x7fc97c0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.432+0000 7fc981dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c0cd240 con 0x7fc97c15c4b0 2023-07-06T21:41:00.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.432+0000 7fc981dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c160e40 con 0x7fc97c15c0e0 2023-07-06T21:41:00.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.432+0000 7fc981dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c0cccc0 con 0x7fc97c1608f0 2023-07-06T21:41:00.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc97c15c4b0 0x7fc97c070b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48330/0 (socket says 172.21.15.84:48330) 2023-07-06T21:41:00.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97bfff700 1 -- 172.21.15.84:0/3964765536 learned_addr learned my addr 172.21.15.84:0/3964765536 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:00.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97affd700 1 --2- 172.21.15.84:0/3964765536 >> v2:172.21.15.107:3300/0 conn(0x7fc97c1608f0 0x7fc97c0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97a7fc700 1 -- 172.21.15.84:0/3964765536 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821201374 0 0) 0x7fc97c0cd240 con 0x7fc97c15c4b0 2023-07-06T21:41:00.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97a7fc700 1 -- 172.21.15.84:0/3964765536 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9640035b0 con 0x7fc97c15c4b0 2023-07-06T21:41:00.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97a7fc700 1 -- 172.21.15.84:0/3964765536 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756080037 0 0) 0x7fc9640035b0 con 0x7fc97c15c4b0 2023-07-06T21:41:00.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97affd700 1 -- 172.21.15.84:0/3964765536 >> v1:172.21.15.107:6789/0 conn(0x7fc97c15c0e0 legacy=0x7fc97c0ce790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97affd700 1 -- 172.21.15.84:0/3964765536 >> v1:172.21.15.84:6789/0 conn(0x7fc97c15c4b0 legacy=0x7fc97c070b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.433+0000 7fc97affd700 1 -- 172.21.15.84:0/3964765536 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c0ccf60 con 0x7fc97c1608f0 2023-07-06T21:41:00.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc97affd700 1 --2- 172.21.15.84:0/3964765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c1608f0 0x7fc97c0c9610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc970009840 tx=0x7fc970009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=157e09523534410c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc97a7fc700 1 -- 172.21.15.84:0/3964765536 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9740019f0 con 0x7fc97c1608f0 2023-07-06T21:41:00.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc97a7fc700 1 -- 172.21.15.84:0/3964765536 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970004030 con 0x7fc97c1608f0 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc97a7fc700 1 -- 172.21.15.84:0/3964765536 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700043c0 con 0x7fc97c1608f0 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc981dff700 1 -- 172.21.15.84:0/3964765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c1608f0 msgr2=0x7fc97c0c9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc981dff700 1 --2- 172.21.15.84:0/3964765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c1608f0 0x7fc97c0c9610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc970009840 tx=0x7fc970009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc981dff700 1 -- 172.21.15.84:0/3964765536 shutdown_connections 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc981dff700 1 --2- 172.21.15.84:0/3964765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c1608f0 0x7fc97c0c9610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.434+0000 7fc981dff700 1 -- 172.21.15.84:0/3964765536 >> 172.21.15.84:0/3964765536 conn(0x7fc97c0cfc50 msgr2=0x7fc97c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.435+0000 7fc981dff700 1 -- 172.21.15.84:0/3964765536 shutdown_connections 2023-07-06T21:41:00.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.435+0000 7fc981dff700 1 -- 172.21.15.84:0/3964765536 wait complete. 2023-07-06T21:41:00.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.436+0000 7fc981dff700 1 Processor -- start 2023-07-06T21:41:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.436+0000 7fc981dff700 1 -- start start 2023-07-06T21:41:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc981dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c15c0e0 0x7fc97c1f2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc981dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c4b0 0x7fc97c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc981dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 0x7fc97c158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc981dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c6fd0 con 0x7fc97c1608f0 2023-07-06T21:41:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc981dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c63c0 con 0x7fc97c15c0e0 2023-07-06T21:41:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc981dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c0c66a0 con 0x7fc97c15c4b0 2023-07-06T21:41:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.437+0000 7fc97bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 0x7fc97c158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 0x7fc97c158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53280/0 (socket says 172.21.15.84:53280) 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 -- 172.21.15.84:0/2555032545 learned_addr learned my addr 172.21.15.84:0/2555032545 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c15c0e0 0x7fc97c1f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97affd700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c4b0 0x7fc97c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 -- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c4b0 msgr2=0x7fc97c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c4b0 0x7fc97c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 -- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c15c0e0 msgr2=0x7fc97c1f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c15c0e0 0x7fc97c1f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 -- 172.21.15.84:0/2555032545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9700094a0 con 0x7fc97c1608f0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc97bfff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 0x7fc97c158740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc97400cbe0 tx=0x7fc97400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc963fff700 1 -- 172.21.15.84:0/2555032545 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97400ab80 con 0x7fc97c1608f0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc963fff700 1 -- 172.21.15.84:0/2555032545 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97400ad00 con 0x7fc97c1608f0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc963fff700 1 -- 172.21.15.84:0/2555032545 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974007310 con 0x7fc97c1608f0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c0c9cd0 con 0x7fc97c1608f0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.438+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c0c6c60 con 0x7fc97c1608f0 2023-07-06T21:41:00.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.441+0000 7fc963fff700 1 -- 172.21.15.84:0/2555032545 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc974007470 con 0x7fc97c1608f0 2023-07-06T21:41:00.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.441+0000 7fc963fff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc96806ace0 0x7fc96806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.441+0000 7fc963fff700 1 -- 172.21.15.84:0/2555032545 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc9740890e0 con 0x7fc97c1608f0 2023-07-06T21:41:00.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.442+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc96806ace0 0x7fc96806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.442+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc96806ace0 0x7fc96806d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc96c000f60 tx=0x7fc96c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.444+0000 7fc995688b80 1 Processor -- start 2023-07-06T21:41:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.444+0000 7fc995688b80 1 -- start start 2023-07-06T21:41:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc995688b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7567320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a758d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7591440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc995688b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a746bc60 con 0x5568a7566f50 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a74be9f0 con 0x5568a758ed80 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a74b7e80 con 0x5568a7564740 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7567320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.458 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7567320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53284/0 (socket says 172.21.15.84:53284) 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 -- 172.21.15.84:0/3634414707 learned_addr learned my addr 172.21.15.84:0/3634414707 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 -- 172.21.15.84:0/3634414707 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 msgr2=0x5568a7591440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7591440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 -- 172.21.15.84:0/3634414707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 msgr2=0x5568a758d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a758d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 -- 172.21.15.84:0/3634414707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a73395e0 con 0x5568a7566f50 2023-07-06T21:41:00.459 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.445+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7567320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007c10 tx=0x7fc96c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d7c9c592a42076 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc961ffb700 1 -- 172.21.15.84:0/3634414707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c003ea0 con 0x5568a7566f50 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc961ffb700 1 -- 172.21.15.84:0/3634414707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c021450 con 0x5568a7566f50 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc961ffb700 1 -- 172.21.15.84:0/3634414707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0217c0 con 0x5568a7566f50 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 -- 172.21.15.84:0/3634414707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 msgr2=0x5568a7567320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7567320 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007c10 tx=0x7fc96c011000 comp rx=0 tx=0).stop 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 -- 172.21.15.84:0/3634414707 shutdown_connections 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7591440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7567320 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 --2- 172.21.15.84:0/3634414707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a758d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 -- 172.21.15.84:0/3634414707 >> 172.21.15.84:0/3634414707 conn(0x5568a755bfc0 msgr2=0x5568a75ac1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 -- 172.21.15.84:0/3634414707 shutdown_connections 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.446+0000 7fc995688b80 1 -- 172.21.15.84:0/3634414707 wait complete. 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.447+0000 7fc995688b80 1 -- start start 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.447+0000 7fc995688b80 1 Processor -- start 2023-07-06T21:41:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.448+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7585cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.448+0000 7fc995688b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a75a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.448+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7587320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.448+0000 7fc995688b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a7457a70 con 0x5568a7566f50 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.448+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a7457770 con 0x5568a758ed80 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.448+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a733b8d0 con 0x5568a7564740 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc995688b80 10 client.0 _ll_register_callbacks cb 0x5568a745d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a75a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a75a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53296/0 (socket says 172.21.15.84:53296) 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 -- 172.21.15.84:0/299510309 learned_addr learned my addr 172.21.15.84:0/299510309 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97bfff700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7587320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7585cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 -- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 msgr2=0x5568a7585cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7585cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 -- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 msgr2=0x5568a7587320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7587320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 -- 172.21.15.84:0/299510309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c00f040 con 0x5568a7566f50 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7585cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97bfff700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7587320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc97affd700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a75a5610 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc9700090b0 tx=0x7fc970004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0eaf819e10c0b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.449+0000 7fc95a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc95a7fc700 1 -- 172.21.15.84:0/299510309 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc97001f070 con 0x5568a7566f50 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568a7559780 con 0x5568a7566f50 2023-07-06T21:41:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc95a7fc700 1 -- 172.21.15.84:0/299510309 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc970003970 con 0x5568a7566f50 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc95a7fc700 1 -- 172.21.15.84:0/299510309 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc970003b30 con 0x5568a7566f50 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 10 client.5511 fetch_fsmap learned FSMap version 15 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5568a72e5d90 con 0x5568a7566f50 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc95a7fc700 1 -- 172.21.15.84:0/299510309 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5568a7559780 con 0x5568a7566f50 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc95a7fc700 1 -- 172.21.15.84:0/299510309 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc970018070 con 0x5568a7566f50 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 10 client.5511 fetch_fsmap finished waiting for FSMap version 15 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 10 client.5511 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T21:41:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 20 client.5511 populate_metadata read hostname 'smithi084' 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75a9d40 0x5568a75aa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 4 client.5511 mds_command: new command op to 5319 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.450+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 --> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5568a74b0e60 con 0x5568a75a9d40 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.451+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75a9d40 0x5568a75aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.451+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75a9d40 0x5568a75aa110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4385a9f2ea9b8374 server_cookie=d87ef22a8aa12e58 in_seq=0 out_seq=0 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.451+0000 7fc95a7fc700 10 client.5511 ms_handle_connect on v2:172.21.15.84:6835/687288152 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc95a7fc700 1 -- 172.21.15.84:0/299510309 <== mds.0 v2:172.21.15.84:6835/687288152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5568a74b0e60 con 0x5568a75a9d40 2023-07-06T21:41:00.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc95a7fc700 10 client.5511 handle_command_reply: tid=0 2023-07-06T21:41:00.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 client.5511 shutdown 2023-07-06T21:41:00.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 msgr2=0x5568a75a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a75a5610 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc9700090b0 tx=0x7fc970004750 comp rx=0 tx=0).stop 2023-07-06T21:41:00.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 shutdown_connections 2023-07-06T21:41:00.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75a9d40 0x5568a75aa110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7587320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7585cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc95a7fc700 0 client.5511 ms_handle_reset on v2:172.21.15.84:6835/687288152 2023-07-06T21:41:00.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 --2- 172.21.15.84:0/299510309 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a75a5610 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 >> 172.21.15.84:0/299510309 conn(0x5568a755bfc0 msgr2=0x5568a7566120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 shutdown_connections 2023-07-06T21:41:00.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.452+0000 7fc995688b80 1 -- 172.21.15.84:0/299510309 wait complete. 2023-07-06T21:41:00.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.453+0000 7fc995688b80 20 client.5511 trim_cache size 0 max 16384 2023-07-06T21:41:00.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.471+0000 7fc995688b80 1 Processor -- start 2023-07-06T21:41:00.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.471+0000 7fc995688b80 1 -- start start 2023-07-06T21:41:00.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7564740 0x5568a75592c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc995688b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7568400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a758ed80 0x5568a7560420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc995688b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a75ac620 con 0x5568a7566f50 2023-07-06T21:41:00.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a75ac7b0 con 0x5568a7564740 2023-07-06T21:41:00.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a75a2bb0 con 0x5568a758ed80 2023-07-06T21:41:00.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7568400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7568400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53312/0 (socket says 172.21.15.84:53312) 2023-07-06T21:41:00.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97affd700 1 -- 172.21.15.84:0/3671788539 learned_addr learned my addr 172.21.15.84:0/3671788539 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:00.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97affd700 1 -- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a758ed80 msgr2=0x5568a7560420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97affd700 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a758ed80 0x5568a7560420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7564740 0x5568a75592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.472+0000 7fc97affd700 1 -- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7564740 msgr2=0x5568a75592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc97affd700 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7564740 0x5568a75592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc97affd700 1 -- 172.21.15.84:0/3671788539 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568a72cec10 con 0x5568a7566f50 2023-07-06T21:41:00.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7564740 0x5568a75592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc97affd700 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7568400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc970004540 tx=0x7fc97001c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626665955796493f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc9627fc700 1 -- 172.21.15.84:0/3671788539 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970003b70 con 0x5568a7566f50 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc9627fc700 1 -- 172.21.15.84:0/3671788539 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970011630 con 0x5568a7566f50 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.473+0000 7fc9627fc700 1 -- 172.21.15.84:0/3671788539 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700119c0 con 0x5568a7566f50 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 -- 172.21.15.84:0/3671788539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 msgr2=0x5568a7568400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7568400 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc970004540 tx=0x7fc97001c070 comp rx=0 tx=0).stop 2023-07-06T21:41:00.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 -- 172.21.15.84:0/3671788539 shutdown_connections 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a758ed80 0x5568a7560420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a7564740 0x5568a75592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 --2- 172.21.15.84:0/3671788539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7568400 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 -- 172.21.15.84:0/3671788539 >> 172.21.15.84:0/3671788539 conn(0x5568a755bfc0 msgr2=0x5568a7585ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 -- 172.21.15.84:0/3671788539 shutdown_connections 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 -- 172.21.15.84:0/3671788539 wait complete. 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.474+0000 7fc995688b80 1 -- start start 2023-07-06T21:41:00.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.475+0000 7fc995688b80 1 Processor -- start 2023-07-06T21:41:00.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7564b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7591da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7610140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a7610680 con 0x5568a7566f50 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568a75fbf90 con 0x5568a758ed80 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5568a75650a0 con 0x5568a7564740 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc995688b80 10 client.0 _ll_register_callbacks cb 0x5568a745d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7591da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7591da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53328/0 (socket says 172.21.15.84:53328) 2023-07-06T21:41:00.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 -- 172.21.15.84:0/1522663603 learned_addr learned my addr 172.21.15.84:0/1522663603 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7564b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97bfff700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7610140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 -- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 msgr2=0x5568a7564b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7564b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 -- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 msgr2=0x5568a7610140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7610140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 -- 172.21.15.84:0/1522663603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5568a72e5d90 con 0x5568a7566f50 2023-07-06T21:41:00.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7564b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97bfff700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7610140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc97affd700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7591da0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc970009fd0 tx=0x7fc970008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9b0c8f5240748b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.476+0000 7fc95a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc95a7fc700 1 -- 172.21.15.84:0/1522663603 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970024880 con 0x5568a7566f50 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc95a7fc700 1 -- 172.21.15.84:0/1522663603 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc970000e40 con 0x5568a7566f50 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc96c021450 con 0x5568a7566f50 2023-07-06T21:41:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc95a7fc700 1 -- 172.21.15.84:0/1522663603 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9700119d0 con 0x5568a7566f50 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 10 client.5517 fetch_fsmap learned FSMap version 15 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x5568a730c200 con 0x5568a7566f50 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc95a7fc700 1 -- 172.21.15.84:0/1522663603 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c021450 con 0x5568a7566f50 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc95a7fc700 1 -- 172.21.15.84:0/1522663603 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc970045020 con 0x5568a7566f50 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 10 client.5517 fetch_fsmap finished waiting for FSMap version 15 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 10 client.5517 resolve_mds: resolved name 'a' to daemon mds.a 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 20 client.5517 populate_metadata read hostname 'smithi084' 2023-07-06T21:41:00.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75fb7a0 0x5568a75660f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 4 client.5517 mds_command: new command op to 5319 tid=0[{"prefix": "status"}] 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 --> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5568a7473000 con 0x5568a75fb7a0 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.477+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75fb7a0 0x5568a75660f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc97b7fe700 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75fb7a0 0x5568a75660f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3423d1ea66c66611 server_cookie=a1dd93bb2ca81847 in_seq=0 out_seq=0 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc95a7fc700 10 client.5517 ms_handle_connect on v2:172.21.15.84:6835/687288152 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc95a7fc700 1 -- 172.21.15.84:0/1522663603 <== mds.0 v2:172.21.15.84:6835/687288152 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5568a7473000 con 0x5568a75fb7a0 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc95a7fc700 10 client.5517 handle_command_reply: tid=0 2023-07-06T21:41:00.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 client.5517 shutdown 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 msgr2=0x5568a7591da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7591da0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc970009fd0 tx=0x7fc970008030 comp rx=0 tx=0).stop 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 shutdown_connections 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:6835/687288152,v1:172.21.15.84:6837/687288152] conn(0x5568a75fb7a0 0x5568a75660f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5568a758ed80 0x5568a7610140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5568a7564740 0x5568a7564b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc95a7fc700 0 client.5517 ms_handle_reset on v2:172.21.15.84:6835/687288152 2023-07-06T21:41:00.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 --2- 172.21.15.84:0/1522663603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5568a7566f50 0x5568a7591da0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.478+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 >> 172.21.15.84:0/1522663603 conn(0x5568a755bfc0 msgr2=0x5568a75a2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 shutdown_connections 2023-07-06T21:41:00.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc995688b80 1 -- 172.21.15.84:0/1522663603 wait complete. 2023-07-06T21:41:00.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc995688b80 20 client.5517 trim_cache size 0 max 16384 2023-07-06T21:41:00.489 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c2762452-5842-4ccc-b0b9-8dbf6ac718c9", 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5319, 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-07-06T21:41:00.490 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.8189064019999996, 2023-07-06T21:41:00.491 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 15, 2023-07-06T21:41:00.491 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 37, 2023-07-06T21:41:00.491 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 37, 2023-07-06T21:41:00.491 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 16.119866765000001 2023-07-06T21:41:00.491 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:41:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc96806ace0 msgr2=0x7fc96806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc96806ace0 0x7fc96806d190 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc96c000f60 tx=0x7fc96c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 msgr2=0x7fc97c158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 0x7fc97c158740 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc97400cbe0 tx=0x7fc97400a060 comp rx=0 tx=0).stop 2023-07-06T21:41:00.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 shutdown_connections 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc96806ace0 0x7fc96806d190 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc97c1608f0 0x7fc97c158740 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c15c0e0 0x7fc97c1f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 --2- 172.21.15.84:0/2555032545 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c4b0 0x7fc97c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 >> 172.21.15.84:0/2555032545 conn(0x7fc97c0cfc50 msgr2=0x7fc97c070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 shutdown_connections 2023-07-06T21:41:00.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.479+0000 7fc981dff700 1 -- 172.21.15.84:0/2555032545 wait complete. 2023-07-06T21:41:00.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:41:00.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.691+0000 7f9b907ff700 1 Processor -- start 2023-07-06T21:41:00.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.691+0000 7f9b907ff700 1 -- start start 2023-07-06T21:41:00.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.692+0000 7f9b907ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9b880cfc60 0x7f9b88167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.692+0000 7f9b907ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880cd910 con 0x7f9b880d3710 2023-07-06T21:41:00.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.692+0000 7f9b907ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b880d0a80 con 0x7f9b880b83c0 2023-07-06T21:41:00.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.692+0000 7f9b907ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b880b8790 con 0x7f9b880cfc60 2023-07-06T21:41:00.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.692+0000 7f9b8dd9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9b880d3710 0x7f9b880cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:48344/0 (socket says 172.21.15.84:48344) 2023-07-06T21:41:00.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/1156270833 learned_addr learned my addr 172.21.15.84:0/1156270833 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8ed9c700 1 --2- 172.21.15.84:0/1156270833 >> v2:172.21.15.107:3300/0 conn(0x7f9b880cfc60 0x7f9b88167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94697452 0 0) 0x7f9b78001110 con 0x7f9b880d3710 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b740035b0 con 0x7f9b880d3710 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3545680642 0 0) 0x7f9b740035b0 con 0x7f9b880d3710 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 >> v2:172.21.15.107:3300/0 conn(0x7f9b880cfc60 msgr2=0x7f9b88167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 --2- 172.21.15.84:0/1156270833 >> v2:172.21.15.107:3300/0 conn(0x7f9b880cfc60 0x7f9b88167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 >> v1:172.21.15.107:6789/0 conn(0x7f9b880b83c0 legacy=0x7f9b880cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b880b4e70 con 0x7f9b880d3710 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f9b78001800 con 0x7f9b880d3710 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 >> v1:172.21.15.84:6789/0 conn(0x7f9b880d3710 legacy=0x7f9b880cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74003a00 0x7f9b74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b740043f0 0x7f9b740048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.693+0000 7f9b8d599700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b74005150 0x7f9b7400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880b8790 con 0x7f9b740043f0 2023-07-06T21:41:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7400c080 con 0x7f9b74003a00 2023-07-06T21:41:00.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7400c240 con 0x7f9b74005150 2023-07-06T21:41:00.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b740043f0 0x7f9b740048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8e59b700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74003a00 0x7f9b74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8ed9c700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b74005150 0x7f9b7400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b74005150 msgr2=0x7f9b7400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b74005150 0x7f9b7400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74003a00 msgr2=0x7f9b74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:00.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b74003a00 0x7f9b74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/1156270833 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740035b0 con 0x7f9b740043f0 2023-07-06T21:41:00.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8ed9c700 1 -- 172.21.15.84:0/1156270833 reap_dead start 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.694+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b740043f0 0x7f9b740048b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c210 tx=0x7f9b74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae2b867e9e9428a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800cdb0 con 0x7f9b740043f0 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78005960 con 0x7f9b740043f0 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b8d599700 1 -- 172.21.15.84:0/1156270833 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78005d10 con 0x7f9b740043f0 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 -- 172.21.15.84:0/1156270833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b740043f0 msgr2=0x7f9b740048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b740043f0 0x7f9b740048b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800c210 tx=0x7f9b74005880 comp rx=0 tx=0).stop 2023-07-06T21:41:00.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 -- 172.21.15.84:0/1156270833 shutdown_connections 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 --2- 172.21.15.84:0/1156270833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b740043f0 0x7f9b740048b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 -- 172.21.15.84:0/1156270833 >> 172.21.15.84:0/1156270833 conn(0x7f9b8806b350 msgr2=0x7f9b8806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 -- 172.21.15.84:0/1156270833 shutdown_connections 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.695+0000 7f9b907ff700 1 -- 172.21.15.84:0/1156270833 wait complete. 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.697+0000 7f9b907ff700 1 Processor -- start 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.697+0000 7f9b907ff700 1 -- start start 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b907ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b740043f0 0x7f9b8815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b907ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 0x7f9b880c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b907ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b880cb470 0x7f9b880cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b907ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b880c9000 con 0x7f9b880c8650 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b907ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8815e030 con 0x7f9b740043f0 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b907ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8815e310 con 0x7f9b880cb470 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b8dd9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 0x7f9b880c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b8dd9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 0x7f9b880c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53350/0 (socket says 172.21.15.84:53350) 2023-07-06T21:41:00.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.698+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/2016350450 learned_addr learned my addr 172.21.15.84:0/2016350450 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8ed9c700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b880cb470 0x7f9b880cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b880cb470 msgr2=0x7f9b880cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b880cb470 0x7f9b880cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8e59b700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b740043f0 0x7f9b8815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b740043f0 msgr2=0x7f9b8815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b740043f0 0x7f9b8815aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8dd9a700 1 -- 172.21.15.84:0/2016350450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8815e5f0 con 0x7f9b880c8650 2023-07-06T21:41:00.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b8dd9a700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 0x7f9b880c8ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c050 tx=0x7f9b78001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b78005ce0 con 0x7f9b880c8650 2023-07-06T21:41:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78005e40 con 0x7f9b880c8650 2023-07-06T21:41:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780072f0 con 0x7f9b880c8650 2023-07-06T21:41:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8815e8d0 con 0x7f9b880c8650 2023-07-06T21:41:00.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.699+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8815b430 con 0x7f9b880c8650 2023-07-06T21:41:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.700+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8815ef20 con 0x7f9b880c8650 2023-07-06T21:41:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.701+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9b78007450 con 0x7f9b880c8650 2023-07-06T21:41:00.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.703+0000 7f9b7effd700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9b6c06ace0 0x7f9b6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:00.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.703+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b7808a530 con 0x7f9b880c8650 2023-07-06T21:41:00.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.703+0000 7f9b8e59b700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9b6c06ace0 0x7f9b6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:00.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.704+0000 7f9b8e59b700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9b6c06ace0 0x7f9b6c06d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b88068f30 tx=0x7f9b84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:00.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.707+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9b78055790 con 0x7f9b880c8650 2023-07-06T21:41:00.947 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.945+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9b880cd890 con 0x7f9b880c8650 2023-07-06T21:41:00.948 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.945+0000 7f9b7effd700 1 -- 172.21.15.84:0/2016350450 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9b78058db0 con 0x7f9b880c8650 2023-07-06T21:41:00.948 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:41:00.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9b6c06ace0 msgr2=0x7f9b6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9b6c06ace0 0x7f9b6c06d190 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9b88068f30 tx=0x7f9b84009040 comp rx=0 tx=0).stop 2023-07-06T21:41:00.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 msgr2=0x7f9b880c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:00.950 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 0x7f9b880c8ac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400c050 tx=0x7f9b78001b90 comp rx=0 tx=0).stop 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 shutdown_connections 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9b6c06ace0 0x7f9b6c06d190 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b880cb470 0x7f9b880cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b740043f0 0x7f9b8815aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 --2- 172.21.15.84:0/2016350450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b880c8650 0x7f9b880c8ac0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 >> 172.21.15.84:0/2016350450 conn(0x7f9b8806b350 msgr2=0x7f9b880b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:00.951 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 shutdown_connections 2023-07-06T21:41:00.952 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:00.948+0000 7f9b907ff700 1 -- 172.21.15.84:0/2016350450 wait complete. 2023-07-06T21:41:00.968 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:41:00.969 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:41:00.969 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:41:00.969 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:41:00.970 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:00.970 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:41:00.970 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:41:00.971 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:41:00.971 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:00.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:00.972 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:41:01.016 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:41:01.016 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:41:01.016 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:41:01.016 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:01.017 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:01.018 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:41:01.018 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:01.018 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:01.018 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:41:01.018 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:01.018 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42141sec preferred_lft 42141sec 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.019 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:41:01.020 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:41:01.021 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4492:bbff:fe16:eebb/64 scope link 2023-07-06T21:41:01.021 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:01.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:01.024 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:41:01.128 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:41:01.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:01.131 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:41:01.177 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:41:01.177 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:41:01.177 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:41:01.177 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:41:01.177 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:41:01.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:01.179 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:41:01.179 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:41:01.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:41:01.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:41:01.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:41:01.180 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:41:01.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:01.386 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:41:01.429 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:41:01.429 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:41:01.430 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:01.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:01.430 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:41:01.536 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:41:01.537 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:41:01.537 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:01.579 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:01.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:41:01.580 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:01.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:41:01.769 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:41:01.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:41:01.836 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:41:01.838 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:41:01.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:01.841 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:41:01.885 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:41:01.886 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:41:01.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:41:01.998 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:41:01.998 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:41:02.065 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:41:02.067 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:41:02.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:02.068 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:41:02.096 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:41:02.094+0000 7f1585be75c0 -1 init, newargv = 0x558d6ff60db0 newargc=15 2023-07-06T21:41:02.097 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[147170]: starting ceph client 2023-07-06T21:41:02.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[147170]: starting fuse 2023-07-06T21:41:02.116 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:41:02.116 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:41:02.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:41:02.117 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:41:02.117 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:41:02.118 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:41:02.118 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:41:02.118 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:02.119 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:41:02.119 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:41:02.119 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:41:02.120 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:41:02.120 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:02.121 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:41:02.121 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:41:02.121 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:41:02.122 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:41:02.122 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:02.122 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:41:02.123 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:41:02.123 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:41:02.124 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:41:02.125 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:41:02.125 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:41:02.126 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:41:02.126 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:41:02.127 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:02.127 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:41:02.127 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:41:02.297 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.147170.asok 2023-07-06T21:41:02.475 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.147170.asok 2023-07-06T21:41:02.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:02.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.147170.asok status 2023-07-06T21:41:02.697 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:41:02.697 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:41:02.698 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:41:02.698 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:41:02.698 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:41:02.698 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:41:02.699 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:41:02.699 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "147170", 2023-07-06T21:41:02.699 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:41:02.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:02.699 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:41:02.700 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:41:02.700 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5368, 2023-07-06T21:41:02.700 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:41:02.700 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:41:02.700 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:41:02.701 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 5368 2023-07-06T21:41:02.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:02.701 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:41:02.701 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:41:02.702 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2023-07-06T21:41:02.702 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2243037093 2023-07-06T21:41:02.703 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:41:02.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:02.703 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:41:02.703 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:41:02.704 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2023-07-06T21:41:02.704 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2243037093 2023-07-06T21:41:02.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:02.705 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.5368 172.21.15.84:0/2243037093", 2023-07-06T21:41:02.705 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "172.21.15.84:0/2243037093", 2023-07-06T21:41:02.705 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:41:02.705 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 15, 2023-07-06T21:41:02.706 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 37, 2023-07-06T21:41:02.706 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:41:02.706 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:41:02.707 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:41:02.707 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:41:02.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:02.712 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:41:02.712 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:02.756 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:41:02.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:02.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:02.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:02.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2023-07-06T21:41:03.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.059+0000 7f407e7ff700 1 Processor -- start 2023-07-06T21:41:03.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.059+0000 7f407e7ff700 1 -- start start 2023-07-06T21:41:03.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.060+0000 7f407e7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4078160ac0 0x7f40780cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:03.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.060+0000 7f407e7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40780cfe20 0x7f407815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:03.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.060+0000 7f407e7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40780ceef0 con 0x7f40780d0ea0 2023-07-06T21:41:03.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.060+0000 7f407e7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780cf450 con 0x7f40780cfe20 2023-07-06T21:41:03.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.060+0000 7f407e7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40780cf120 con 0x7f4078160ac0 2023-07-06T21:41:03.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f407cd9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40780cfe20 0x7f407815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:03.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f4077fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f40780d0ea0 0x7f40780d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53962/0 (socket says 172.21.15.84:53962) 2023-07-06T21:41:03.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f4077fff700 1 -- 172.21.15.84:0/1246662198 learned_addr learned my addr 172.21.15.84:0/1246662198 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:03.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f40777fe700 1 --2- 172.21.15.84:0/1246662198 >> v2:172.21.15.107:3300/0 conn(0x7f4078160ac0 0x7f40780cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:03.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f407cd9c700 1 -- 172.21.15.84:0/1246662198 >> v2:172.21.15.107:3300/0 conn(0x7f4078160ac0 msgr2=0x7f40780cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:03.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f407cd9c700 1 --2- 172.21.15.84:0/1246662198 >> v2:172.21.15.107:3300/0 conn(0x7f4078160ac0 0x7f40780cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f407cd9c700 1 -- 172.21.15.84:0/1246662198 >> v1:172.21.15.84:6789/0 conn(0x7f40780d0ea0 legacy=0x7f40780d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:03.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f407cd9c700 1 -- 172.21.15.84:0/1246662198 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40780cd420 con 0x7f40780cfe20 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f4076ffd700 1 -- 172.21.15.84:0/1246662198 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511693806 0 0) 0x7f40780ceef0 con 0x7f40780d0ea0 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f40777fe700 1 --2- 172.21.15.84:0/1246662198 >> v2:172.21.15.107:3300/0 conn(0x7f4078160ac0 0x7f40780cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f407cd9c700 1 --2- 172.21.15.84:0/1246662198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f407815fee0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40780b5870 tx=0x7f407000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cad76148d9246bc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.061+0000 7f4076ffd700 1 -- 172.21.15.84:0/1246662198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407000e070 con 0x7f40780cfe20 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f4076ffd700 1 -- 172.21.15.84:0/1246662198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407000bbd0 con 0x7f40780cfe20 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f4076ffd700 1 -- 172.21.15.84:0/1246662198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070012430 con 0x7f40780cfe20 2023-07-06T21:41:03.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 -- 172.21.15.84:0/1246662198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 msgr2=0x7f407815fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 --2- 172.21.15.84:0/1246662198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f407815fee0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40780b5870 tx=0x7f407000b330 comp rx=0 tx=0).stop 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 -- 172.21.15.84:0/1246662198 shutdown_connections 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 --2- 172.21.15.84:0/1246662198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f407815fee0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 --2- 172.21.15.84:0/1246662198 >> v2:172.21.15.107:3300/0 conn(0x7f4078160ac0 0x7f40780cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 -- 172.21.15.84:0/1246662198 >> 172.21.15.84:0/1246662198 conn(0x7f40780715c0 msgr2=0x7f4078071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 -- 172.21.15.84:0/1246662198 shutdown_connections 2023-07-06T21:41:03.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.062+0000 7f407e7ff700 1 -- 172.21.15.84:0/1246662198 wait complete. 2023-07-06T21:41:03.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.064+0000 7f407e7ff700 1 Processor -- start 2023-07-06T21:41:03.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.064+0000 7f407e7ff700 1 -- start start 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f4078152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078160ac0 0x7f407815edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f407e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f40780b8420 0x7f4078153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f407e7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40781533e0 con 0x7f40780cfe20 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f407e7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780b7da0 con 0x7f4078160ac0 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f407e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40780b77e0 con 0x7f40780b8420 2023-07-06T21:41:03.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f4077fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f4078152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f4077fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f4078152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38622/0 (socket says 172.21.15.84:38622) 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f4077fff700 1 -- 172.21.15.84:0/546591237 learned_addr learned my addr 172.21.15.84:0/546591237 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.065+0000 7f40777fe700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078160ac0 0x7f407815edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f407cd9c700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f40780b8420 0x7f4078153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4077fff700 1 -- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f40780b8420 msgr2=0x7f4078153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4077fff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f40780b8420 0x7f4078153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4077fff700 1 -- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078160ac0 msgr2=0x7f407815edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4077fff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078160ac0 0x7f407815edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4077fff700 1 -- 172.21.15.84:0/546591237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4070009a70 con 0x7f40780cfe20 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4077fff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f4078152ac0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f40680010e0 tx=0x7f406800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40680074e0 con 0x7f40780cfe20 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4068007640 con 0x7f40780cfe20 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068005a70 con 0x7f40780cfe20 2023-07-06T21:41:03.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40780b7ac0 con 0x7f40780cfe20 2023-07-06T21:41:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.066+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407815e8b0 con 0x7f40780cfe20 2023-07-06T21:41:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.067+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407804e1d0 con 0x7f40780cfe20 2023-07-06T21:41:03.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.068+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4068005bd0 con 0x7f40780cfe20 2023-07-06T21:41:03.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.069+0000 7f4074ff9700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.069+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f40680889c0 con 0x7f40780cfe20 2023-07-06T21:41:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.070+0000 7f40777fe700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.070+0000 7f40777fe700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f406406ace0 0x7f406406d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f40780b6510 tx=0x7f406c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:03.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.073+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f40680539d0 con 0x7f40780cfe20 2023-07-06T21:41:03.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.272+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) v1 -- 0x7f407815a740 con 0x7f40780cfe20 2023-07-06T21:41:03.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.989+0000 7f4074ff9700 1 -- 172.21.15.84:0/546591237 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v38) v1 ==== 124+0+0 (secure 0 0 0) 0x7f40680170a0 con 0x7f40780cfe20 2023-07-06T21:41:03.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.994+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f406406ace0 msgr2=0x7f406406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.994+0000 7f407e7ff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f406406ace0 0x7f406406d190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f40780b6510 tx=0x7f406c008040 comp rx=0 tx=0).stop 2023-07-06T21:41:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.994+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 msgr2=0x7f4078152ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.994+0000 7f407e7ff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f4078152ac0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f40680010e0 tx=0x7f406800ca30 comp rx=0 tx=0).stop 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.995+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 shutdown_connections 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.995+0000 7f407e7ff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f406406ace0 0x7f406406d190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.995+0000 7f407e7ff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f40780b8420 0x7f4078153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.995+0000 7f407e7ff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40780cfe20 0x7f4078152ac0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.995+0000 7f407e7ff700 1 --2- 172.21.15.84:0/546591237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4078160ac0 0x7f407815edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.995+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 >> 172.21.15.84:0/546591237 conn(0x7f40780715c0 msgr2=0x7f4078159c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.996+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 shutdown_connections 2023-07-06T21:41:03.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:03.996+0000 7f407e7ff700 1 -- 172.21.15.84:0/546591237 wait complete. 2023-07-06T21:41:04.003 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'data_new' created 2023-07-06T21:41:04.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2023-07-06T21:41:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.218+0000 7f0c1b5ff700 1 Processor -- start 2023-07-06T21:41:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.218+0000 7f0c1b5ff700 1 -- start start 2023-07-06T21:41:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.219+0000 7f0c1b5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0c140b81c0 0x7f0c140cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.219+0000 7f0c1b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b1a80 0x7f0c140cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.219+0000 7f0c1b5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140d1830 con 0x7f0c140ceda0 2023-07-06T21:41:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.219+0000 7f0c1b5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140cf330 con 0x7f0c140b81c0 2023-07-06T21:41:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.219+0000 7f0c1b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140d1270 con 0x7f0c140b1a80 2023-07-06T21:41:04.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.220+0000 7f0c19b9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0c140ceda0 0x7f0c1415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53970/0 (socket says 172.21.15.84:53970) 2023-07-06T21:41:04.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.220+0000 7f0c19b9c700 1 -- 172.21.15.84:0/3885928437 learned_addr learned my addr 172.21.15.84:0/3885928437 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:04.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.220+0000 7f0c18b9a700 1 --2- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b1a80 0x7f0c140cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.220+0000 7f0c1939b700 1 --2- 172.21.15.84:0/3885928437 >> v2:172.21.15.107:3300/0 conn(0x7f0c140b81c0 0x7f0c140cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.220+0000 7f0c0bfff700 1 -- 172.21.15.84:0/3885928437 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3160497087 0 0) 0x7f0c140d1830 con 0x7f0c140ceda0 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.220+0000 7f0c0bfff700 1 -- 172.21.15.84:0/3885928437 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c000035b0 con 0x7f0c140ceda0 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c1939b700 1 -- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b1a80 msgr2=0x7f0c140cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c0bfff700 1 -- 172.21.15.84:0/3885928437 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2806503636 0 0) 0x7f0c000035b0 con 0x7f0c140ceda0 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c1939b700 1 --2- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b1a80 0x7f0c140cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c1939b700 1 -- 172.21.15.84:0/3885928437 >> v1:172.21.15.84:6789/0 conn(0x7f0c140ceda0 legacy=0x7f0c1415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c1939b700 1 -- 172.21.15.84:0/3885928437 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140d1550 con 0x7f0c140b81c0 2023-07-06T21:41:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c1939b700 1 --2- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140b81c0 0x7f0c140cf600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009840 tx=0x7f0c10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6fc6dc771278aa5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c0bfff700 1 -- 172.21.15.84:0/3885928437 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0019f0 con 0x7f0c140b81c0 2023-07-06T21:41:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c0bfff700 1 -- 172.21.15.84:0/3885928437 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10004030 con 0x7f0c140b81c0 2023-07-06T21:41:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.221+0000 7f0c0bfff700 1 -- 172.21.15.84:0/3885928437 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100043c0 con 0x7f0c140b81c0 2023-07-06T21:41:04.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140b81c0 msgr2=0x7f0c140cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140b81c0 0x7f0c140cf600 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c10009840 tx=0x7f0c10009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/3885928437 shutdown_connections 2023-07-06T21:41:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140b81c0 0x7f0c140cf600 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/3885928437 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b1a80 0x7f0c140cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/3885928437 >> 172.21.15.84:0/3885928437 conn(0x7f0c1406b2b0 msgr2=0x7f0c1406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/3885928437 shutdown_connections 2023-07-06T21:41:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.222+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/3885928437 wait complete. 2023-07-06T21:41:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.224+0000 7f0c1b5ff700 1 Processor -- start 2023-07-06T21:41:04.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.224+0000 7f0c1b5ff700 1 -- start start 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1b5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 0x7f0c140c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b81c0 0x7f0c140cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1b5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140ceda0 0x7f0c140b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1b5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140cb010 con 0x7f0c140b1a80 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1b5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140ccd50 con 0x7f0c140ceda0 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140cb4a0 con 0x7f0c140b81c0 2023-07-06T21:41:04.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 0x7f0c140c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 0x7f0c140c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38636/0 (socket says 172.21.15.84:38636) 2023-07-06T21:41:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 -- 172.21.15.84:0/1932178593 learned_addr learned my addr 172.21.15.84:0/1932178593 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 -- 172.21.15.84:0/1932178593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b81c0 msgr2=0x7f0c140cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b81c0 0x7f0c140cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 -- 172.21.15.84:0/1932178593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140ceda0 msgr2=0x7f0c140b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140ceda0 0x7f0c140b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.225+0000 7f0c1939b700 1 -- 172.21.15.84:0/1932178593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c100094a0 con 0x7f0c140b1a80 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.226+0000 7f0c1939b700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 0x7f0c140c9e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0c100090b0 tx=0x7f0c10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.226+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10003dd0 con 0x7f0c140b1a80 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.226+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1001f070 con 0x7f0c140b1a80 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.226+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10011870 con 0x7f0c140b1a80 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.226+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c140ca400 con 0x7f0c140b1a80 2023-07-06T21:41:04.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.226+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c140ca9c0 con 0x7f0c140b1a80 2023-07-06T21:41:04.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.227+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1404e1d0 con 0x7f0c140b1a80 2023-07-06T21:41:04.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.228+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0c100119d0 con 0x7f0c140b1a80 2023-07-06T21:41:04.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.229+0000 7f0c09ffb700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bf806ace0 0x7f0bf806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.229+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f0c10021070 con 0x7f0c140b1a80 2023-07-06T21:41:04.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.229+0000 7f0c18b9a700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bf806ace0 0x7f0bf806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.230+0000 7f0c18b9a700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bf806ace0 0x7f0bf806d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006b20 tx=0x7f0c04006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:04.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.234+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0c10055b50 con 0x7f0c140b1a80 2023-07-06T21:41:04.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.474+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) v1 -- 0x7f0c141e6db0 con 0x7f0c140b1a80 2023-07-06T21:41:04.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.481+0000 7f0c09ffb700 1 -- 172.21.15.84:0/1932178593 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 7 to fsmap v16) v1 ==== 131+0+0 (secure 0 0 0) 0x7f0c10059170 con 0x7f0c140b1a80 2023-07-06T21:41:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bf806ace0 msgr2=0x7f0bf806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bf806ace0 0x7f0bf806d190 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0c04006b20 tx=0x7f0c04006ab0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 msgr2=0x7f0c140c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 0x7f0c140c9e20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0c100090b0 tx=0x7f0c10004750 comp rx=0 tx=0).stop 2023-07-06T21:41:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 shutdown_connections 2023-07-06T21:41:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0bf806ace0 0x7f0bf806d190 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c140ceda0 0x7f0c140b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0c140b1a80 0x7f0c140c9e20 secure :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0c100090b0 tx=0x7f0c10004750 comp rx=0 tx=0).stop 2023-07-06T21:41:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 --2- 172.21.15.84:0/1932178593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c140b81c0 0x7f0c140cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.486+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 >> 172.21.15.84:0/1932178593 conn(0x7f0c1406b2b0 msgr2=0x7f0c140b2810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:04.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.493+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 shutdown_connections 2023-07-06T21:41:04.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.494+0000 7f0c1b5ff700 1 -- 172.21.15.84:0/1932178593 wait complete. 2023-07-06T21:41:04.498 INFO:teuthology.orchestra.run.smithi084.stderr:added data pool 7 to fsmap 2023-07-06T21:41:04.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:41:04.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.707+0000 7f3d66bff700 1 Processor -- start 2023-07-06T21:41:04.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.707+0000 7f3d66bff700 1 -- start start 2023-07-06T21:41:04.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.708+0000 7f3d66bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d60069e90 0x7f3d6006a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.708+0000 7f3d66bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6006a820 con 0x7f3d600c92c0 2023-07-06T21:41:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.708+0000 7f3d66bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d600c3f80 con 0x7f3d60158830 2023-07-06T21:41:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.708+0000 7f3d66bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600c4260 con 0x7f3d60069e90 2023-07-06T21:41:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d6519c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3d600c92c0 0x7f3d600c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53982/0 (socket says 172.21.15.84:53982) 2023-07-06T21:41:04.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d6519c700 1 -- 172.21.15.84:0/2776787750 learned_addr learned my addr 172.21.15.84:0/2776787750 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5ffff700 1 --2- 172.21.15.84:0/2776787750 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d60069e90 0x7f3d6006a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982505546 0 0) 0x7f3d58001490 con 0x7f3d600c92c0 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d500035b0 con 0x7f3d600c92c0 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 513742730 0 0) 0x7f3d600c3f80 con 0x7f3d60158830 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d58001490 con 0x7f3d60158830 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5ffff700 1 -- 172.21.15.84:0/2776787750 >> v1:172.21.15.107:6789/0 conn(0x7f3d60158830 legacy=0x7f3d600c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5ffff700 1 -- 172.21.15.84:0/2776787750 >> v1:172.21.15.84:6789/0 conn(0x7f3d600c92c0 legacy=0x7f3d600c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.709+0000 7f3d5ffff700 1 -- 172.21.15.84:0/2776787750 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600c4540 con 0x7f3d60069e90 2023-07-06T21:41:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d5ffff700 1 --2- 172.21.15.84:0/2776787750 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d60069e90 0x7f3d6006a2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00aa10 tx=0x7f3d4c00ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0d65e42f0369d03 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00cb70 con 0x7f3d60069e90 2023-07-06T21:41:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4c00ccd0 con 0x7f3d60069e90 2023-07-06T21:41:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d5f7fe700 1 -- 172.21.15.84:0/2776787750 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c018620 con 0x7f3d60069e90 2023-07-06T21:41:04.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d66bff700 1 -- 172.21.15.84:0/2776787750 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d60069e90 msgr2=0x7f3d6006a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d66bff700 1 --2- 172.21.15.84:0/2776787750 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d60069e90 0x7f3d6006a2e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00aa10 tx=0x7f3d4c00ad20 comp rx=0 tx=0).stop 2023-07-06T21:41:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d66bff700 1 -- 172.21.15.84:0/2776787750 shutdown_connections 2023-07-06T21:41:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d66bff700 1 --2- 172.21.15.84:0/2776787750 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d60069e90 0x7f3d6006a2e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.710+0000 7f3d66bff700 1 -- 172.21.15.84:0/2776787750 >> 172.21.15.84:0/2776787750 conn(0x7f3d600cf3d0 msgr2=0x7f3d600cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.711+0000 7f3d66bff700 1 -- 172.21.15.84:0/2776787750 shutdown_connections 2023-07-06T21:41:04.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.711+0000 7f3d66bff700 1 -- 172.21.15.84:0/2776787750 wait complete. 2023-07-06T21:41:04.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.712+0000 7f3d66bff700 1 Processor -- start 2023-07-06T21:41:04.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.712+0000 7f3d66bff700 1 -- start start 2023-07-06T21:41:04.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.713+0000 7f3d66bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 0x7f3d601f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.713+0000 7f3d66bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d600c92c0 0x7f3d600c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.713+0000 7f3d66bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 0x7f3d600c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.713+0000 7f3d66bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600ab940 con 0x7f3d60158830 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.713+0000 7f3d66bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600c6dc0 con 0x7f3d60069e90 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.713+0000 7f3d66bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600abbe0 con 0x7f3d600c92c0 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 0x7f3d600c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6499b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 0x7f3d601f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6499b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 0x7f3d601f1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:33232/0 (socket says 172.21.15.84:33232) 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6499b700 1 -- 172.21.15.84:0/369134288 learned_addr learned my addr 172.21.15.84:0/369134288 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 0x7f3d600c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38640/0 (socket says 172.21.15.84:38640) 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d5ffff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d600c92c0 0x7f3d600c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 -- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d600c92c0 msgr2=0x7f3d600c7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d600c92c0 0x7f3d600c7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 -- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 msgr2=0x7f3d601f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 0x7f3d601f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 -- 172.21.15.84:0/369134288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c00a640 con 0x7f3d60158830 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6499b700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 0x7f3d601f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.714+0000 7f3d6519c700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 0x7f3d600c7fc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800cc00 tx=0x7f3d5800cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.715+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800aa80 con 0x7f3d60158830 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.715+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5800abe0 con 0x7f3d60158830 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.715+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d600abec0 con 0x7f3d60158830 2023-07-06T21:41:04.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.715+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58012400 con 0x7f3d60158830 2023-07-06T21:41:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.715+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6014bda0 con 0x7f3d60158830 2023-07-06T21:41:04.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.716+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6015d2d0 con 0x7f3d60158830 2023-07-06T21:41:04.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.717+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3d58012560 con 0x7f3d60158830 2023-07-06T21:41:04.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.718+0000 7f3d5d7fa700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d4406ace0 0x7f3d4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:04.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.718+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3d58089080 con 0x7f3d60158830 2023-07-06T21:41:04.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.718+0000 7f3d6499b700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d4406ace0 0x7f3d4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:04.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.719+0000 7f3d6499b700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d4406ace0 0x7f3d4406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3d54001220 tx=0x7f3d54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:04.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.722+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3d58054050 con 0x7f3d60158830 2023-07-06T21:41:04.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.963+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d600ac100 con 0x7f3d60158830 2023-07-06T21:41:04.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.964+0000 7f3d5d7fa700 1 -- 172.21.15.84:0/369134288 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f3d600ac100 con 0x7f3d60158830 2023-07-06T21:41:04.966 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:41:04.476261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:41:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d4406ace0 msgr2=0x7f3d4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d4406ace0 0x7f3d4406d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3d54001220 tx=0x7f3d54009040 comp rx=0 tx=0).stop 2023-07-06T21:41:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 msgr2=0x7f3d600c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 0x7f3d600c7fc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3d5800cc00 tx=0x7f3d5800cf10 comp rx=0 tx=0).stop 2023-07-06T21:41:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 shutdown_connections 2023-07-06T21:41:04.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3d4406ace0 0x7f3d4406d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3d60158830 0x7f3d600c7fc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d60069e90 0x7f3d601f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 --2- 172.21.15.84:0/369134288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d600c92c0 0x7f3d600c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.966+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 >> 172.21.15.84:0/369134288 conn(0x7f3d600cf3d0 msgr2=0x7f3d600c9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.967+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 shutdown_connections 2023-07-06T21:41:04.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:04.967+0000 7f3d66bff700 1 -- 172.21.15.84:0/369134288 wait complete. 2023-07-06T21:41:04.971 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-07-06T21:41:04.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:05.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.178+0000 7f859e3ff700 1 Processor -- start 2023-07-06T21:41:05.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.178+0000 7f859e3ff700 1 -- start start 2023-07-06T21:41:05.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.179+0000 7f859e3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 0x7f85980cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f859e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f859815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f859e3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85980d0990 con 0x7f85980d3580 2023-07-06T21:41:05.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f859e3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85980d3af0 con 0x7f85980b8530 2023-07-06T21:41:05.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f859e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85980cd9d0 con 0x7f85980d1580 2023-07-06T21:41:05.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f859c99c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f859815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f8597fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 0x7f85980cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f8597fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 0x7f85980cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:33246/0 (socket says 172.21.15.84:33246) 2023-07-06T21:41:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.180+0000 7f8597fff700 1 -- 172.21.15.84:0/3281284031 learned_addr learned my addr 172.21.15.84:0/3281284031 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f85977fe700 1 -- 172.21.15.84:0/3281284031 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362007489 0 0) 0x7f85980d0990 con 0x7f85980d3580 2023-07-06T21:41:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f85977fe700 1 -- 172.21.15.84:0/3281284031 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85840035b0 con 0x7f85980d3580 2023-07-06T21:41:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f859c99c700 1 -- 172.21.15.84:0/3281284031 >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 msgr2=0x7f85980cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f859c99c700 1 --2- 172.21.15.84:0/3281284031 >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 0x7f85980cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f859c99c700 1 -- 172.21.15.84:0/3281284031 >> v1:172.21.15.107:6789/0 conn(0x7f85980d3580 legacy=0x7f85980d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f859c99c700 1 -- 172.21.15.84:0/3281284031 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85980cd410 con 0x7f85980d1580 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f8597fff700 1 --2- 172.21.15.84:0/3281284031 >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 0x7f85980cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.181+0000 7f859c99c700 1 --2- 172.21.15.84:0/3281284031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f859815c8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8590009d90 tx=0x7f859000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36fb741611178c0e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f85977fe700 1 -- 172.21.15.84:0/3281284031 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859000e070 con 0x7f85980d1580 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f85977fe700 1 -- 172.21.15.84:0/3281284031 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859000bb20 con 0x7f85980d1580 2023-07-06T21:41:05.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f85977fe700 1 -- 172.21.15.84:0/3281284031 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590012430 con 0x7f85980d1580 2023-07-06T21:41:05.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 -- 172.21.15.84:0/3281284031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 msgr2=0x7f859815c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3281284031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f859815c8e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8590009d90 tx=0x7f859000b330 comp rx=0 tx=0).stop 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 -- 172.21.15.84:0/3281284031 shutdown_connections 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3281284031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f859815c8e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3281284031 >> v2:172.21.15.107:3300/0 conn(0x7f85980b8530 0x7f85980cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 -- 172.21.15.84:0/3281284031 >> 172.21.15.84:0/3281284031 conn(0x7f85980715c0 msgr2=0x7f8598071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 -- 172.21.15.84:0/3281284031 shutdown_connections 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.182+0000 7f859e3ff700 1 -- 172.21.15.84:0/3281284031 wait complete. 2023-07-06T21:41:05.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.184+0000 7f859e3ff700 1 Processor -- start 2023-07-06T21:41:05.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.184+0000 7f859e3ff700 1 -- start start 2023-07-06T21:41:05.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f859e3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 0x7f859815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f859e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f85980ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f859e3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85980d3580 0x7f85980b6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f859e3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85980cbf90 con 0x7f85980b8530 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f859e3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85980cb310 con 0x7f85980d3580 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f859e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85980cb5f0 con 0x7f85980d1580 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f8597fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 0x7f859815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f8597fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 0x7f859815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38646/0 (socket says 172.21.15.84:38646) 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.185+0000 7f8597fff700 1 -- 172.21.15.84:0/3502902250 learned_addr learned my addr 172.21.15.84:0/3502902250 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f8597fff700 1 -- 172.21.15.84:0/3502902250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 msgr2=0x7f85980ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f8597fff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f85980ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f8597fff700 1 -- 172.21.15.84:0/3502902250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85980d3580 msgr2=0x7f85980b6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f8597fff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85980d3580 0x7f85980b6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f8597fff700 1 -- 172.21.15.84:0/3502902250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588009500 con 0x7f85980b8530 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f8597fff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 0x7f859815ac30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f858800ebb0 tx=0x7f858800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588005a20 con 0x7f85980b8530 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588005b80 con 0x7f85980b8530 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8590009a70 con 0x7f85980b8530 2023-07-06T21:41:05.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800c910 con 0x7f85980b8530 2023-07-06T21:41:05.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.186+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85980cbbb0 con 0x7f85980b8530 2023-07-06T21:41:05.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.187+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8598073a50 con 0x7f85980b8530 2023-07-06T21:41:05.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.188+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f858800ca70 con 0x7f85980b8530 2023-07-06T21:41:05.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.189+0000 7f85957fa700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f857806ace0 0x7f857806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.189+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f85880899c0 con 0x7f85980b8530 2023-07-06T21:41:05.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.189+0000 7f858ffff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f857806ace0 0x7f857806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.190+0000 7f858ffff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f857806ace0 0x7f857806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f85800010e0 tx=0x7f85800079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:05.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.194+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f85880548d0 con 0x7f85980b8530 2023-07-06T21:41:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.393+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f859804e240 con 0x7f85980b8530 2023-07-06T21:41:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.394+0000 7f85957fa700 1 -- 172.21.15.84:0/3502902250 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+20143 (secure 0 0 0) 0x7f8588057ef0 con 0x7f85980b8530 2023-07-06T21:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":39,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:04.475888+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f857806ace0 msgr2=0x7f857806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f857806ace0 0x7f857806d190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f85800010e0 tx=0x7f85800079e0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 msgr2=0x7f859815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 0x7f859815ac30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f858800ebb0 tx=0x7f858800ef70 comp rx=0 tx=0).stop 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 shutdown_connections 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f857806ace0 0x7f857806d190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85980d3580 0x7f85980b6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85980b8530 0x7f859815ac30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 --2- 172.21.15.84:0/3502902250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85980d1580 0x7f85980ca680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 >> 172.21.15.84:0/3502902250 conn(0x7f85980715c0 msgr2=0x7f8598073340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:05.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 shutdown_connections 2023-07-06T21:41:05.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.397+0000 7f859e3ff700 1 -- 172.21.15.84:0/3502902250 wait complete. 2023-07-06T21:41:05.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:05.418 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2023-07-06T21:41:05.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:05.467 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2023-07-06T21:41:05.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-07-06T21:41:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.717+0000 7fdb1a3ff700 1 Processor -- start 2023-07-06T21:41:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.717+0000 7fdb1a3ff700 1 -- start start 2023-07-06T21:41:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.718+0000 7fdb1a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 0x7fdb140cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.718+0000 7fdb1a3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdb140cfe20 0x7fdb1415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.718+0000 7fdb1a3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb140ceef0 con 0x7fdb140d0ea0 2023-07-06T21:41:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.718+0000 7fdb1a3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb140cf450 con 0x7fdb140cfe20 2023-07-06T21:41:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.718+0000 7fdb1a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140cf120 con 0x7fdb14160ac0 2023-07-06T21:41:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 0x7fdb140cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 0x7fdb140cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:55770/0 (socket says 172.21.15.84:55770) 2023-07-06T21:41:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 -- 172.21.15.84:0/701576488 learned_addr learned my addr 172.21.15.84:0/701576488 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb1899c700 1 --2- 172.21.15.84:0/701576488 >> v2:172.21.15.107:3300/0 conn(0x7fdb140cfe20 0x7fdb1415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 -- 172.21.15.84:0/701576488 >> v2:172.21.15.107:3300/0 conn(0x7fdb140cfe20 msgr2=0x7fdb1415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 --2- 172.21.15.84:0/701576488 >> v2:172.21.15.107:3300/0 conn(0x7fdb140cfe20 0x7fdb1415fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 -- 172.21.15.84:0/701576488 >> v1:172.21.15.107:6789/0 conn(0x7fdb140d0ea0 legacy=0x7fdb140d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.719+0000 7fdb137fe700 1 -- 172.21.15.84:0/701576488 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb140cd420 con 0x7fdb14160ac0 2023-07-06T21:41:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb137fe700 1 --2- 172.21.15.84:0/701576488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 0x7fdb140cf8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098f0 tx=0x7fdb08009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c7b2fe1051cbb60 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb12ffd700 1 -- 172.21.15.84:0/701576488 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08011830 con 0x7fdb14160ac0 2023-07-06T21:41:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb12ffd700 1 -- 172.21.15.84:0/701576488 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb08011990 con 0x7fdb14160ac0 2023-07-06T21:41:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb12ffd700 1 -- 172.21.15.84:0/701576488 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb08011d20 con 0x7fdb14160ac0 2023-07-06T21:41:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/701576488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 msgr2=0x7fdb140cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/701576488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 0x7fdb140cf8e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdb080098f0 tx=0x7fdb08009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/701576488 shutdown_connections 2023-07-06T21:41:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/701576488 >> v2:172.21.15.107:3300/0 conn(0x7fdb140cfe20 0x7fdb1415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/701576488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb14160ac0 0x7fdb140cf8e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.720+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/701576488 >> 172.21.15.84:0/701576488 conn(0x7fdb140715c0 msgr2=0x7fdb14071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:05.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.721+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/701576488 shutdown_connections 2023-07-06T21:41:05.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.721+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/701576488 wait complete. 2023-07-06T21:41:05.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.722+0000 7fdb1a3ff700 1 Processor -- start 2023-07-06T21:41:05.730 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.722+0000 7fdb1a3ff700 1 -- start start 2023-07-06T21:41:05.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.723+0000 7fdb1a3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 0x7fdb1413fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.723+0000 7fdb1a3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb14160ac0 0x7fdb140caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.723+0000 7fdb1a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb140b5c90 0x7fdb140cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.723+0000 7fdb1a3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140b8c30 con 0x7fdb140cfe20 2023-07-06T21:41:05.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.723+0000 7fdb1a3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140c95a0 con 0x7fdb14160ac0 2023-07-06T21:41:05.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.723+0000 7fdb1a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb140b8670 con 0x7fdb140b5c90 2023-07-06T21:41:05.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 0x7fdb1413fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 0x7fdb1413fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38660/0 (socket says 172.21.15.84:38660) 2023-07-06T21:41:05.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 -- 172.21.15.84:0/2576262018 learned_addr learned my addr 172.21.15.84:0/2576262018 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:05.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb137fe700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb14160ac0 0x7fdb140caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb1899c700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb140b5c90 0x7fdb140cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 -- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb140b5c90 msgr2=0x7fdb140cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb140b5c90 0x7fdb140cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 -- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb14160ac0 msgr2=0x7fdb140caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:05.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb14160ac0 0x7fdb140caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:05.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 -- 172.21.15.84:0/2576262018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb08009550 con 0x7fdb140cfe20 2023-07-06T21:41:05.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.724+0000 7fdb13fff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 0x7fdb1413fec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c4d0 tx=0x7fdb0400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:05.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.725+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040052d0 con 0x7fdb140cfe20 2023-07-06T21:41:05.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.725+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb04005430 con 0x7fdb140cfe20 2023-07-06T21:41:05.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.725+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb040038f0 con 0x7fdb140cfe20 2023-07-06T21:41:05.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.725+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb140b8950 con 0x7fdb140cfe20 2023-07-06T21:41:05.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.725+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb140c91d0 con 0x7fdb140cfe20 2023-07-06T21:41:05.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.726+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb140c9360 con 0x7fdb140cfe20 2023-07-06T21:41:05.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.727+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb04003a50 con 0x7fdb140cfe20 2023-07-06T21:41:05.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.727+0000 7fdb10ff9700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdb0006ace0 0x7fdb0006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:05.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.728+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdb040889e0 con 0x7fdb140cfe20 2023-07-06T21:41:05.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.728+0000 7fdb137fe700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdb0006ace0 0x7fdb0006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:05.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.728+0000 7fdb137fe700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdb0006ace0 0x7fdb0006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb14160ef0 tx=0x7fdb08008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:05.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:05.732+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb140c9360 con 0x7fdb140cfe20 2023-07-06T21:41:06.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.008+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7fdb140c9360 con 0x7fdb140cfe20 2023-07-06T21:41:06.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.011+0000 7fdb10ff9700 1 -- 172.21.15.84:0/2576262018 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v11) v1 ==== 210+0+0 (secure 0 0 0) 0x7fdb140c9360 con 0x7fdb140cfe20 2023-07-06T21:41:06.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.013+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdb0006ace0 msgr2=0x7fdb0006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.013+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdb0006ace0 0x7fdb0006d190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdb14160ef0 tx=0x7fdb08008030 comp rx=0 tx=0).stop 2023-07-06T21:41:06.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.013+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 msgr2=0x7fdb1413fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.013+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 0x7fdb1413fec0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c4d0 tx=0x7fdb0400c7e0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 shutdown_connections 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdb0006ace0 0x7fdb0006d190 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb140b5c90 0x7fdb140cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb140cfe20 0x7fdb1413fec0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 --2- 172.21.15.84:0/2576262018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb14160ac0 0x7fdb140caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 >> 172.21.15.84:0/2576262018 conn(0x7fdb140715c0 msgr2=0x7fdb1415f080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 shutdown_connections 2023-07-06T21:41:06.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.014+0000 7fdb1a3ff700 1 -- 172.21.15.84:0/2576262018 wait complete. 2023-07-06T21:41:06.018 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:41:06.034 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:06.034 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:06.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:06.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.309+0000 7fa404dff700 1 Processor -- start 2023-07-06T21:41:06.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.309+0000 7fa404dff700 1 -- start start 2023-07-06T21:41:06.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.310+0000 7fa404dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4000b1260 0x7fa4000cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.310+0000 7fa404dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa40006ba90 0x7fa4000b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa404dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa400160960 con 0x7fa40006ee90 2023-07-06T21:41:06.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa404dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40006a5b0 con 0x7fa4000b1260 2023-07-06T21:41:06.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa404dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa400160e90 con 0x7fa40006ba90 2023-07-06T21:41:06.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4000b1260 0x7fa4000cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa4000b1260 0x7fa4000cef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38664/0 (socket says 172.21.15.84:38664) 2023-07-06T21:41:06.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 -- 172.21.15.84:0/688899803 learned_addr learned my addr 172.21.15.84:0/688899803 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:06.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3f7fff700 1 --2- 172.21.15.84:0/688899803 >> v2:172.21.15.107:3300/0 conn(0x7fa40006ba90 0x7fa4000b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 -- 172.21.15.84:0/688899803 >> v2:172.21.15.107:3300/0 conn(0x7fa40006ba90 msgr2=0x7fa4000b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:06.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 --2- 172.21.15.84:0/688899803 >> v2:172.21.15.107:3300/0 conn(0x7fa40006ba90 0x7fa4000b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 -- 172.21.15.84:0/688899803 >> v1:172.21.15.107:6789/0 conn(0x7fa40006ee90 legacy=0x7fa40006f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3fe59c700 1 -- 172.21.15.84:0/688899803 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40015c820 con 0x7fa4000b1260 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.311+0000 7fa3f7fff700 1 --2- 172.21.15.84:0/688899803 >> v2:172.21.15.107:3300/0 conn(0x7fa40006ba90 0x7fa4000b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa3fe59c700 1 --2- 172.21.15.84:0/688899803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4000b1260 0x7fa4000cef60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b1ec0 tx=0x7fa3f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f2be3cf985e67a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa3fdd9b700 1 -- 172.21.15.84:0/688899803 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000b860 con 0x7fa4000b1260 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa3fdd9b700 1 -- 172.21.15.84:0/688899803 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f000b9c0 con 0x7fa4000b1260 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa3fdd9b700 1 -- 172.21.15.84:0/688899803 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f000bd50 con 0x7fa4000b1260 2023-07-06T21:41:06.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 -- 172.21.15.84:0/688899803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4000b1260 msgr2=0x7fa4000cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 --2- 172.21.15.84:0/688899803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4000b1260 0x7fa4000cef60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b1ec0 tx=0x7fa3f0009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 -- 172.21.15.84:0/688899803 shutdown_connections 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 --2- 172.21.15.84:0/688899803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4000b1260 0x7fa4000cef60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 --2- 172.21.15.84:0/688899803 >> v2:172.21.15.107:3300/0 conn(0x7fa40006ba90 0x7fa4000b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 -- 172.21.15.84:0/688899803 >> 172.21.15.84:0/688899803 conn(0x7fa4000cc260 msgr2=0x7fa4000cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 -- 172.21.15.84:0/688899803 shutdown_connections 2023-07-06T21:41:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.312+0000 7fa404dff700 1 -- 172.21.15.84:0/688899803 wait complete. 2023-07-06T21:41:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.314+0000 7fa404dff700 1 Processor -- start 2023-07-06T21:41:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.314+0000 7fa404dff700 1 -- start start 2023-07-06T21:41:06.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa404dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa40006ba90 0x7fa40014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa404dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 0x7fa4000add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa404dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4000b1260 0x7fa4000b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa404dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000ae8e0 con 0x7fa40006ee90 2023-07-06T21:41:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa404dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40014e6b0 con 0x7fa4000b1260 2023-07-06T21:41:06.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa404dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4000ae600 con 0x7fa40006ba90 2023-07-06T21:41:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 0x7fa4000add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa3f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 0x7fa4000add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38674/0 (socket says 172.21.15.84:38674) 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.315+0000 7fa3f7fff700 1 -- 172.21.15.84:0/3877200162 learned_addr learned my addr 172.21.15.84:0/3877200162 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3fe59c700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa40006ba90 0x7fa40014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3fed9d700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4000b1260 0x7fa4000b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f7fff700 1 -- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa40006ba90 msgr2=0x7fa40014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f7fff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa40006ba90 0x7fa40014e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f7fff700 1 -- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4000b1260 msgr2=0x7fa4000b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f7fff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4000b1260 0x7fa4000b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f7fff700 1 -- 172.21.15.84:0/3877200162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f0009550 con 0x7fa40006ee90 2023-07-06T21:41:06.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3fed9d700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4000b1260 0x7fa4000b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:06.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f7fff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 0x7fa4000add90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e760 tx=0x7fa3e800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8013610 con 0x7fa40006ee90 2023-07-06T21:41:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8013770 con 0x7fa40006ee90 2023-07-06T21:41:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4001e5f10 con 0x7fa40006ee90 2023-07-06T21:41:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.316+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8013b20 con 0x7fa40006ee90 2023-07-06T21:41:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.317+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4001e64d0 con 0x7fa40006ee90 2023-07-06T21:41:06.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.317+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4001e5d70 con 0x7fa40006ee90 2023-07-06T21:41:06.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.319+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa3e8013c80 con 0x7fa40006ee90 2023-07-06T21:41:06.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.320+0000 7fa3f6ffd700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa3e006ace0 0x7fa3e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.320+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa3e8089090 con 0x7fa40006ee90 2023-07-06T21:41:06.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.320+0000 7fa3fe59c700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa3e006ace0 0x7fa3e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.321+0000 7fa3fe59c700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa3e006ace0 0x7fa3e006d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f0008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:06.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.324+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa4001e5d70 con 0x7fa40006ee90 2023-07-06T21:41:06.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.522+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4001e5d70 con 0x7fa40006ee90 2023-07-06T21:41:06.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.524+0000 7fa3f6ffd700 1 -- 172.21.15.84:0/3877200162 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7fa4001e5d70 con 0x7fa40006ee90 2023-07-06T21:41:06.526 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:06.527 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:06.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa3e006ace0 msgr2=0x7fa3e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa3e006ace0 0x7fa3e006d190 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0000c00 tx=0x7fa3f0008030 comp rx=0 tx=0).stop 2023-07-06T21:41:06.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 msgr2=0x7fa4000add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 0x7fa4000add90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800e760 tx=0x7fa3e800ea70 comp rx=0 tx=0).stop 2023-07-06T21:41:06.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 shutdown_connections 2023-07-06T21:41:06.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa3e006ace0 0x7fa3e006d190 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4000b1260 0x7fa4000b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa40006ba90 0x7fa40014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 --2- 172.21.15.84:0/3877200162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa40006ee90 0x7fa4000add90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.526+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 >> 172.21.15.84:0/3877200162 conn(0x7fa4000cc260 msgr2=0x7fa4000b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:06.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.527+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 shutdown_connections 2023-07-06T21:41:06.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.527+0000 7fa404dff700 1 -- 172.21.15.84:0/3877200162 wait complete. 2023-07-06T21:41:06.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:06.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:06.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.743+0000 7f3c6c1ff700 1 Processor -- start 2023-07-06T21:41:06.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.743+0000 7f3c6c1ff700 1 -- start start 2023-07-06T21:41:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6c1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3c640b1a80 0x7f3c640d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6c1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640d1830 con 0x7f3c640cfd10 2023-07-06T21:41:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6c1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640cf330 con 0x7f3c640b81c0 2023-07-06T21:41:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6c1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c640d1270 con 0x7f3c640b1a80 2023-07-06T21:41:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6979a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3c640cfd10 0x7f3c640d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53990/0 (socket says 172.21.15.84:53990) 2023-07-06T21:41:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6979a700 1 -- 172.21.15.84:0/867521001 learned_addr learned my addr 172.21.15.84:0/867521001 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.744+0000 7f3c6a79c700 1 --2- 172.21.15.84:0/867521001 >> v2:172.21.15.84:3300/0 conn(0x7f3c640b1a80 0x7f3c640d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c68f99700 1 -- 172.21.15.84:0/867521001 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476759311 0 0) 0x7f3c640d1830 con 0x7f3c640cfd10 2023-07-06T21:41:06.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c68f99700 1 -- 172.21.15.84:0/867521001 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c500035b0 con 0x7f3c640cfd10 2023-07-06T21:41:06.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c6a79c700 1 -- 172.21.15.84:0/867521001 >> v1:172.21.15.107:6789/0 conn(0x7f3c640b81c0 legacy=0x7f3c640cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c6a79c700 1 -- 172.21.15.84:0/867521001 >> v1:172.21.15.84:6789/0 conn(0x7f3c640cfd10 legacy=0x7f3c640d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c6a79c700 1 -- 172.21.15.84:0/867521001 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640d1550 con 0x7f3c640b1a80 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c6a79c700 1 --2- 172.21.15.84:0/867521001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c640d2a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009d90 tx=0x7f3c5c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbba53ac5cb0e73d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c68f99700 1 -- 172.21.15.84:0/867521001 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00e070 con 0x7f3c640b1a80 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c68f99700 1 -- 172.21.15.84:0/867521001 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c00bb20 con 0x7f3c640b1a80 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.745+0000 7f3c68f99700 1 -- 172.21.15.84:0/867521001 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c012430 con 0x7f3c640b1a80 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/867521001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 msgr2=0x7f3c640d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/867521001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c640d2a90 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009d90 tx=0x7f3c5c00b330 comp rx=0 tx=0).stop 2023-07-06T21:41:06.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/867521001 shutdown_connections 2023-07-06T21:41:06.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/867521001 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c640d2a90 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/867521001 >> 172.21.15.84:0/867521001 conn(0x7f3c6406b2b0 msgr2=0x7f3c6406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:06.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/867521001 shutdown_connections 2023-07-06T21:41:06.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.746+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/867521001 wait complete. 2023-07-06T21:41:06.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.748+0000 7f3c6c1ff700 1 Processor -- start 2023-07-06T21:41:06.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.748+0000 7f3c6c1ff700 1 -- start start 2023-07-06T21:41:06.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c6c1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c641e5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c6c1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c640b81c0 0x7f3c640b6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c6c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c640cfd10 0x7f3c641e6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c6c1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c64156900 con 0x7f3c640b1a80 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c6c1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c641e6150 con 0x7f3c640b81c0 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c6c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640b6e90 con 0x7f3c640cfd10 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.749+0000 7f3c69f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c641e5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c641e5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38686/0 (socket says 172.21.15.84:38686) 2023-07-06T21:41:06.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 -- 172.21.15.84:0/1939620700 learned_addr learned my addr 172.21.15.84:0/1939620700 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c6979a700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c640b81c0 0x7f3c640b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c6a79c700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c640cfd10 0x7f3c641e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 -- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c640cfd10 msgr2=0x7f3c641e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c640cfd10 0x7f3c641e6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 -- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c640b81c0 msgr2=0x7f3c640b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c640b81c0 0x7f3c640b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 -- 172.21.15.84:0/1939620700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c009a70 con 0x7f3c640b1a80 2023-07-06T21:41:06.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c69f9b700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c641e5c10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c4e0 tx=0x7f3c6000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:06.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600052d0 con 0x7f3c640b1a80 2023-07-06T21:41:06.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c60005430 con 0x7f3c640b1a80 2023-07-06T21:41:06.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c600038c0 con 0x7f3c640b1a80 2023-07-06T21:41:06.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.750+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c641e6e70 con 0x7f3c640b1a80 2023-07-06T21:41:06.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.751+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c640b7400 con 0x7f3c640b1a80 2023-07-06T21:41:06.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.751+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6404e1d0 con 0x7f3c640b1a80 2023-07-06T21:41:06.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.752+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3c60003a20 con 0x7f3c640b1a80 2023-07-06T21:41:06.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.753+0000 7f3c5a7fc700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c4806ace0 0x7f3c4806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:06.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.754+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3c600889c0 con 0x7f3c640b1a80 2023-07-06T21:41:06.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.754+0000 7f3c6979a700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c4806ace0 0x7f3c4806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:06.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.754+0000 7f3c6979a700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c4806ace0 0x7f3c4806d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3c640ca210 tx=0x7f3c54007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:06.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.758+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3c600538d0 con 0x7f3c640b1a80 2023-07-06T21:41:06.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.953+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c640b4f30 con 0x7f3c640b1a80 2023-07-06T21:41:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.953+0000 7f3c5a7fc700 1 -- 172.21.15.84:0/1939620700 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c60056ef0 con 0x7f3c640b1a80 2023-07-06T21:41:06.956 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:06.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.955+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c4806ace0 msgr2=0x7f3c4806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.955+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c4806ace0 0x7f3c4806d190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3c640ca210 tx=0x7f3c54007dc0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.955+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 msgr2=0x7f3c641e5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:06.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.955+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c641e5c10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3c6000c4e0 tx=0x7f3c6000c7f0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 shutdown_connections 2023-07-06T21:41:06.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c4806ace0 0x7f3c4806d190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c640cfd10 0x7f3c641e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c640b1a80 0x7f3c641e5c10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 --2- 172.21.15.84:0/1939620700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c640b81c0 0x7f3c640b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 >> 172.21.15.84:0/1939620700 conn(0x7f3c6406b2b0 msgr2=0x7f3c640c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 shutdown_connections 2023-07-06T21:41:06.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:06.956+0000 7f3c6c1ff700 1 -- 172.21.15.84:0/1939620700 wait complete. 2023-07-06T21:41:06.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:07.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.171+0000 7f97967ff700 1 Processor -- start 2023-07-06T21:41:07.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.172+0000 7f97967ff700 1 -- start start 2023-07-06T21:41:07.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f97967ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f979015c020 0x7f97900c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f97967ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c460 0x7f9790069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f97967ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f979006cf40 con 0x7f979015c8d0 2023-07-06T21:41:07.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f97967ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f979015d400 con 0x7f979015c020 2023-07-06T21:41:07.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f97967ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979015ce40 con 0x7f979015c460 2023-07-06T21:41:07.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f978f7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c460 0x7f9790069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.173+0000 7f978ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f979015c020 0x7f97900c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f9794d9c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f979015c8d0 0x7f97900703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:59424/0 (socket says 172.21.15.84:59424) 2023-07-06T21:41:07.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f9794d9c700 1 -- 172.21.15.84:0/1670968273 learned_addr learned my addr 172.21.15.84:0/1670968273 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:07.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978effd700 1 -- 172.21.15.84:0/1670968273 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977953055 0 0) 0x7f979006cf40 con 0x7f979015c8d0 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978effd700 1 -- 172.21.15.84:0/1670968273 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97780035b0 con 0x7f979015c8d0 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978f7fe700 1 -- 172.21.15.84:0/1670968273 >> v2:172.21.15.107:3300/0 conn(0x7f979015c020 msgr2=0x7f97900c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978f7fe700 1 --2- 172.21.15.84:0/1670968273 >> v2:172.21.15.107:3300/0 conn(0x7f979015c020 0x7f97900c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978f7fe700 1 -- 172.21.15.84:0/1670968273 >> v1:172.21.15.107:6789/0 conn(0x7f979015c8d0 legacy=0x7f97900703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978f7fe700 1 -- 172.21.15.84:0/1670968273 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979015d120 con 0x7f979015c460 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978ffff700 1 --2- 172.21.15.84:0/1670968273 >> v2:172.21.15.107:3300/0 conn(0x7f979015c020 0x7f97900c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:07.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.174+0000 7f978f7fe700 1 --2- 172.21.15.84:0/1670968273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c460 0x7f9790069e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9784009840 tx=0x7f9784009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4995059186ee62a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:07.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f978effd700 1 -- 172.21.15.84:0/1670968273 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784011830 con 0x7f979015c460 2023-07-06T21:41:07.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f978effd700 1 -- 172.21.15.84:0/1670968273 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9784011990 con 0x7f979015c460 2023-07-06T21:41:07.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f978effd700 1 -- 172.21.15.84:0/1670968273 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784011d20 con 0x7f979015c460 2023-07-06T21:41:07.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 -- 172.21.15.84:0/1670968273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c460 msgr2=0x7f9790069e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 --2- 172.21.15.84:0/1670968273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c460 0x7f9790069e90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9784009840 tx=0x7f9784009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:07.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 -- 172.21.15.84:0/1670968273 shutdown_connections 2023-07-06T21:41:07.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 --2- 172.21.15.84:0/1670968273 >> v2:172.21.15.107:3300/0 conn(0x7f979015c020 0x7f97900c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 --2- 172.21.15.84:0/1670968273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c460 0x7f9790069e90 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 -- 172.21.15.84:0/1670968273 >> 172.21.15.84:0/1670968273 conn(0x7f97900cbf10 msgr2=0x7f97900ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 -- 172.21.15.84:0/1670968273 shutdown_connections 2023-07-06T21:41:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.175+0000 7f97967ff700 1 -- 172.21.15.84:0/1670968273 wait complete. 2023-07-06T21:41:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.177+0000 7f97967ff700 1 Processor -- start 2023-07-06T21:41:07.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.177+0000 7f97967ff700 1 -- start start 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f97967ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f979015c020 0x7f97900b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f97967ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 0x7f97900c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f97967ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c8d0 0x7f97900c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f97967ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900b29b0 con 0x7f979015c460 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f97967ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97900c2c00 con 0x7f979015c020 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f97967ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97900b0c60 con 0x7f979015c8d0 2023-07-06T21:41:07.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f978f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 0x7f97900c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f978f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 0x7f97900c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38694/0 (socket says 172.21.15.84:38694) 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.178+0000 7f978f7fe700 1 -- 172.21.15.84:0/323952354 learned_addr learned my addr 172.21.15.84:0/323952354 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f9794d9c700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c8d0 0x7f97900c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978ffff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f979015c020 0x7f97900b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978f7fe700 1 -- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c8d0 msgr2=0x7f97900c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978f7fe700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c8d0 0x7f97900c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978f7fe700 1 -- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f979015c020 msgr2=0x7f97900b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978f7fe700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f979015c020 0x7f97900b0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978f7fe700 1 -- 172.21.15.84:0/323952354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800094a0 con 0x7f979015c460 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978f7fe700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 0x7f97900c2180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9784009fd0 tx=0x7f9784004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.179+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97840036a0 con 0x7f979015c460 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.180+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f978401e070 con 0x7f979015c460 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.180+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97840094a0 con 0x7f979015c460 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.180+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9784003a60 con 0x7f979015c460 2023-07-06T21:41:07.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.180+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97900b2010 con 0x7f979015c460 2023-07-06T21:41:07.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.181+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97900c8450 con 0x7f979015c460 2023-07-06T21:41:07.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.182+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9784003bc0 con 0x7f979015c460 2023-07-06T21:41:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.183+0000 7f978cff9700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f977c06ace0 0x7f977c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.183+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f978408a4c0 con 0x7f979015c460 2023-07-06T21:41:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.183+0000 7f978ffff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f977c06ace0 0x7f977c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.184+0000 7f978ffff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f977c06ace0 0x7f977c06d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9780009cd0 tx=0x7f9780009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:07.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.188+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f97900c8450 con 0x7f979015c460 2023-07-06T21:41:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.386+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f97900b2af0 con 0x7f979015c460 2023-07-06T21:41:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.387+0000 7f978cff9700 1 -- 172.21.15.84:0/323952354 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f97900b2af0 con 0x7f979015c460 2023-07-06T21:41:07.390 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f977c06ace0 msgr2=0x7f977c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f977c06ace0 0x7f977c06d190 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9780009cd0 tx=0x7f9780009210 comp rx=0 tx=0).stop 2023-07-06T21:41:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 msgr2=0x7f97900c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 0x7f97900c2180 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9784009fd0 tx=0x7f9784004750 comp rx=0 tx=0).stop 2023-07-06T21:41:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 shutdown_connections 2023-07-06T21:41:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f977c06ace0 0x7f977c06d190 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f979015c8d0 0x7f97900c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f979015c020 0x7f97900b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 --2- 172.21.15.84:0/323952354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f979015c460 0x7f97900c2180 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 >> 172.21.15.84:0/323952354 conn(0x7f97900cbf10 msgr2=0x7f97900aced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 shutdown_connections 2023-07-06T21:41:07.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.389+0000 7f97967ff700 1 -- 172.21.15.84:0/323952354 wait complete. 2023-07-06T21:41:07.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.603+0000 7fd86cbff700 1 Processor -- start 2023-07-06T21:41:07.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.604+0000 7fd86cbff700 1 -- start start 2023-07-06T21:41:07.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd86cbff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd8680b81c0 0x7fd8680cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd86cbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8680b1a80 0x7fd8680cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd86cbff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8680d1830 con 0x7fd8680ceda0 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd86cbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680cf330 con 0x7fd8680b1a80 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd86cbff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8680d1270 con 0x7fd8680b81c0 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd85ffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd8680b1a80 0x7fd8680cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd866d9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd8680ceda0 0x7fd86815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:53998/0 (socket says 172.21.15.84:53998) 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd866d9d700 1 -- 172.21.15.84:0/3260242549 learned_addr learned my addr 172.21.15.84:0/3260242549 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.605+0000 7fd86659c700 1 --2- 172.21.15.84:0/3260242549 >> v2:172.21.15.107:3300/0 conn(0x7fd8680b81c0 0x7fd8680cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd85ffff700 1 -- 172.21.15.84:0/3260242549 >> v2:172.21.15.107:3300/0 conn(0x7fd8680b81c0 msgr2=0x7fd8680cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd85ffff700 1 --2- 172.21.15.84:0/3260242549 >> v2:172.21.15.107:3300/0 conn(0x7fd8680b81c0 0x7fd8680cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd85ffff700 1 -- 172.21.15.84:0/3260242549 >> v1:172.21.15.84:6789/0 conn(0x7fd8680ceda0 legacy=0x7fd86815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd85ffff700 1 -- 172.21.15.84:0/3260242549 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680d1550 con 0x7fd8680b1a80 2023-07-06T21:41:07.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd85ffff700 1 --2- 172.21.15.84:0/3260242549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680cfb40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8680b2620 tx=0x7fd850009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a53efd90fffbc62 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd865d9b700 1 -- 172.21.15.84:0/3260242549 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011830 con 0x7fd8680b1a80 2023-07-06T21:41:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd865d9b700 1 -- 172.21.15.84:0/3260242549 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd850011990 con 0x7fd8680b1a80 2023-07-06T21:41:07.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.606+0000 7fd865d9b700 1 -- 172.21.15.84:0/3260242549 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850011d20 con 0x7fd8680b1a80 2023-07-06T21:41:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 -- 172.21.15.84:0/3260242549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 msgr2=0x7fd8680cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 --2- 172.21.15.84:0/3260242549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680cfb40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd8680b2620 tx=0x7fd850009b20 comp rx=0 tx=0).stop 2023-07-06T21:41:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 -- 172.21.15.84:0/3260242549 shutdown_connections 2023-07-06T21:41:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 --2- 172.21.15.84:0/3260242549 >> v2:172.21.15.107:3300/0 conn(0x7fd8680b81c0 0x7fd8680cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 --2- 172.21.15.84:0/3260242549 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680cfb40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 -- 172.21.15.84:0/3260242549 >> 172.21.15.84:0/3260242549 conn(0x7fd86806b2b0 msgr2=0x7fd86806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 -- 172.21.15.84:0/3260242549 shutdown_connections 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.607+0000 7fd86cbff700 1 -- 172.21.15.84:0/3260242549 wait complete. 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.608+0000 7fd86cbff700 1 Processor -- start 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.609+0000 7fd86cbff700 1 -- start start 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.609+0000 7fd86cbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.609+0000 7fd86cbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8680b81c0 0x7fd8680d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86cbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8680ceda0 0x7fd8680c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86cbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680ca4b0 con 0x7fd8680b1a80 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86cbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680caa70 con 0x7fd8680b81c0 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86cbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8680c71e0 con 0x7fd8680ceda0 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38714/0 (socket says 172.21.15.84:38714) 2023-07-06T21:41:07.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 -- 172.21.15.84:0/868196379 learned_addr learned my addr 172.21.15.84:0/868196379 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd85ffff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8680b81c0 0x7fd8680d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd866d9d700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8680ceda0 0x7fd8680c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 -- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8680ceda0 msgr2=0x7fd8680c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8680ceda0 0x7fd8680c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 -- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8680b81c0 msgr2=0x7fd8680d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8680b81c0 0x7fd8680d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 -- 172.21.15.84:0/868196379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500094a0 con 0x7fd8680b1a80 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd866d9d700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8680ceda0 0x7fd8680c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.610+0000 7fd86659c700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680caf00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd85800e840 tx=0x7fd85800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.611+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858013610 con 0x7fd8680b1a80 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.611+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858013770 con 0x7fd8680b1a80 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.611+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8680c74c0 con 0x7fd8680b1a80 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.611+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858013b00 con 0x7fd8680b1a80 2023-07-06T21:41:07.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.611+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8680ccc40 con 0x7fd8680b1a80 2023-07-06T21:41:07.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.612+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8680cb440 con 0x7fd8680b1a80 2023-07-06T21:41:07.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.613+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd858013c60 con 0x7fd8680b1a80 2023-07-06T21:41:07.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.613+0000 7fd85effd700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd84806ace0 0x7fd84806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:07.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.614+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd858089340 con 0x7fd8680b1a80 2023-07-06T21:41:07.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.614+0000 7fd85ffff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd84806ace0 0x7fd84806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:07.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.614+0000 7fd85ffff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd84806ace0 0x7fd84806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8500090b0 tx=0x7fd85001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:07.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.618+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd8580541c0 con 0x7fd8680b1a80 2023-07-06T21:41:07.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.814+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8680d3360 con 0x7fd8680b1a80 2023-07-06T21:41:07.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.814+0000 7fd85effd700 1 -- 172.21.15.84:0/868196379 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8580577e0 con 0x7fd8680b1a80 2023-07-06T21:41:07.817 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:07.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd84806ace0 msgr2=0x7fd84806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd84806ace0 0x7fd84806d190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd8500090b0 tx=0x7fd85001c040 comp rx=0 tx=0).stop 2023-07-06T21:41:07.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 msgr2=0x7fd8680caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:07.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680caf00 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd85800e840 tx=0x7fd85800ec00 comp rx=0 tx=0).stop 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 shutdown_connections 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd84806ace0 0x7fd84806d190 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8680ceda0 0x7fd8680c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8680b1a80 0x7fd8680caf00 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 --2- 172.21.15.84:0/868196379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8680b81c0 0x7fd8680d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 >> 172.21.15.84:0/868196379 conn(0x7fd86806b2b0 msgr2=0x7fd8680b3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:07.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 shutdown_connections 2023-07-06T21:41:07.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:07.817+0000 7fd86cbff700 1 -- 172.21.15.84:0/868196379 wait complete. 2023-07-06T21:41:07.837 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:08.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.033+0000 7fc0725ff700 1 Processor -- start 2023-07-06T21:41:08.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.033+0000 7fc0725ff700 1 -- start start 2023-07-06T21:41:08.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.034+0000 7fc0725ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 0x7fc06c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.034+0000 7fc0725ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 0x7fc06c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.034+0000 7fc0725ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c0d1830 con 0x7fc06c0b81c0 2023-07-06T21:41:08.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.034+0000 7fc0725ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c0cf330 con 0x7fc06c0cfd10 2023-07-06T21:41:08.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.034+0000 7fc0725ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0d1270 con 0x7fc06c0b1a80 2023-07-06T21:41:08.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 0x7fc06c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc063fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 0x7fc06c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 0x7fc06c0d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:55858/0 (socket says 172.21.15.84:55858) 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc063fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 0x7fc06c0d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:33318/0 (socket says 172.21.15.84:33318) 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 -- 172.21.15.84:0/601700213 learned_addr learned my addr 172.21.15.84:0/601700213 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 -- 172.21.15.84:0/601700213 >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 msgr2=0x7fc06c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 --2- 172.21.15.84:0/601700213 >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 0x7fc06c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 -- 172.21.15.84:0/601700213 >> v1:172.21.15.107:6789/0 conn(0x7fc06c0b81c0 legacy=0x7fc06c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc070b9c700 1 -- 172.21.15.84:0/601700213 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c0d1550 con 0x7fc06c0b1a80 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.035+0000 7fc063fff700 1 --2- 172.21.15.84:0/601700213 >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 0x7fc06c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc070b9c700 1 --2- 172.21.15.84:0/601700213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 0x7fc06c0d28c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc064009e50 tx=0x7fc06400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fca1ce59d3733c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc06b7fe700 1 -- 172.21.15.84:0/601700213 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400e070 con 0x7fc06c0b1a80 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc06b7fe700 1 -- 172.21.15.84:0/601700213 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06400b910 con 0x7fc06c0b1a80 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc06b7fe700 1 -- 172.21.15.84:0/601700213 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400bcc0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc0725ff700 1 -- 172.21.15.84:0/601700213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 msgr2=0x7fc06c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc0725ff700 1 --2- 172.21.15.84:0/601700213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 0x7fc06c0d28c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc064009e50 tx=0x7fc06400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc0725ff700 1 -- 172.21.15.84:0/601700213 shutdown_connections 2023-07-06T21:41:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc0725ff700 1 --2- 172.21.15.84:0/601700213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b1a80 0x7fc06c0d28c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc0725ff700 1 --2- 172.21.15.84:0/601700213 >> v2:172.21.15.107:3300/0 conn(0x7fc06c0cfd10 0x7fc06c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.036+0000 7fc0725ff700 1 -- 172.21.15.84:0/601700213 >> 172.21.15.84:0/601700213 conn(0x7fc06c06b2b0 msgr2=0x7fc06c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.037+0000 7fc0725ff700 1 -- 172.21.15.84:0/601700213 shutdown_connections 2023-07-06T21:41:08.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.037+0000 7fc0725ff700 1 -- 172.21.15.84:0/601700213 wait complete. 2023-07-06T21:41:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.038+0000 7fc0725ff700 1 Processor -- start 2023-07-06T21:41:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.038+0000 7fc0725ff700 1 -- start start 2023-07-06T21:41:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc0725ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 0x7fc06c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc0725ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b81c0 0x7fc06c15d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc0725ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0cfd10 0x7fc06c0b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc0725ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c159e10 con 0x7fc06c0b1a80 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc0725ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c15f320 con 0x7fc06c0cfd10 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc0725ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c158e90 con 0x7fc06c0b81c0 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc06bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 0x7fc06c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc06bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 0x7fc06c158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38724/0 (socket says 172.21.15.84:38724) 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.039+0000 7fc06bfff700 1 -- 172.21.15.84:0/1187729836 learned_addr learned my addr 172.21.15.84:0/1187729836 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc063fff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b81c0 0x7fc06c15d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc070b9c700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0cfd10 0x7fc06c0b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc06bfff700 1 -- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b81c0 msgr2=0x7fc06c15d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc06bfff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b81c0 0x7fc06c15d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc06bfff700 1 -- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0cfd10 msgr2=0x7fc06c0b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc06bfff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0cfd10 0x7fc06c0b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc06bfff700 1 -- 172.21.15.84:0/1187729836 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540094a0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc06bfff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 0x7fc06c158950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00c4e0 tx=0x7fc05c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0052d0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c005430 con 0x7fc06c0b1a80 2023-07-06T21:41:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c0038c0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.040+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc064009b30 con 0x7fc06c0b1a80 2023-07-06T21:41:08.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.041+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c15e9f0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.042+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c04e1d0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.043+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc05c003a20 con 0x7fc06c0b1a80 2023-07-06T21:41:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.044+0000 7fc0697fa700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc04c06ace0 0x7fc04c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.044+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc05c0889c0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.044+0000 7fc063fff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc04c06ace0 0x7fc04c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.044+0000 7fc063fff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc04c06ace0 0x7fc04c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0cd320 tx=0x7fc054009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.048+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc05c0538d0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.247+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc06c0b4f30 con 0x7fc06c0b1a80 2023-07-06T21:41:08.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.247+0000 7fc0697fa700 1 -- 172.21.15.84:0/1187729836 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc05c056ef0 con 0x7fc06c0b1a80 2023-07-06T21:41:08.250 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc04c06ace0 msgr2=0x7fc04c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc04c06ace0 0x7fc04c06d190 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0cd320 tx=0x7fc054009210 comp rx=0 tx=0).stop 2023-07-06T21:41:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 msgr2=0x7fc06c158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 0x7fc06c158950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00c4e0 tx=0x7fc05c00c7f0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 shutdown_connections 2023-07-06T21:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc04c06ace0 0x7fc04c06d190 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc06c0cfd10 0x7fc06c0b2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc06c0b1a80 0x7fc06c158950 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 --2- 172.21.15.84:0/1187729836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc06c0b81c0 0x7fc06c15d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 >> 172.21.15.84:0/1187729836 conn(0x7fc06c06b2b0 msgr2=0x7fc06c0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 shutdown_connections 2023-07-06T21:41:08.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.249+0000 7fc0725ff700 1 -- 172.21.15.84:0/1187729836 wait complete. 2023-07-06T21:41:08.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:08.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.464+0000 7f882d7ff700 1 Processor -- start 2023-07-06T21:41:08.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.465+0000 7f882d7ff700 1 -- start start 2023-07-06T21:41:08.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f882d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b81c0 0x7f88280cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f882d7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280d1830 con 0x7f88280ceda0 2023-07-06T21:41:08.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f882d7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8828160d40 con 0x7f88280b1a80 2023-07-06T21:41:08.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f882d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88280cf330 con 0x7f88280b81c0 2023-07-06T21:41:08.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f88277fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f88280ceda0 0x7f8828160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54004/0 (socket says 172.21.15.84:54004) 2023-07-06T21:41:08.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f88277fe700 1 -- 172.21.15.84:0/4216997691 learned_addr learned my addr 172.21.15.84:0/4216997691 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:08.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f8826ffd700 1 --2- 172.21.15.84:0/4216997691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b81c0 0x7f88280cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.466+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754048160 0 0) 0x7f88280d1830 con 0x7f88280ceda0 2023-07-06T21:41:08.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88140035b0 con 0x7f88280ceda0 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241281687 0 0) 0x7f8828160d40 con 0x7f88280b1a80 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88280d1830 con 0x7f88280b1a80 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2200064753 0 0) 0x7f88140035b0 con 0x7f88280ceda0 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f8826ffd700 1 -- 172.21.15.84:0/4216997691 >> v1:172.21.15.107:6789/0 conn(0x7f88280b1a80 legacy=0x7f882815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f8826ffd700 1 -- 172.21.15.84:0/4216997691 >> v1:172.21.15.84:6789/0 conn(0x7f88280ceda0 legacy=0x7f8828160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f8826ffd700 1 -- 172.21.15.84:0/4216997691 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280d1270 con 0x7f88280b81c0 2023-07-06T21:41:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f8826ffd700 1 --2- 172.21.15.84:0/4216997691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b81c0 0x7f88280cf600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8818009870 tx=0x7f8818009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a0259bdb8a0f08 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200019f0 con 0x7f88280b81c0 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8818004030 con 0x7f88280b81c0 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.467+0000 7f88267fc700 1 -- 172.21.15.84:0/4216997691 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88180043c0 con 0x7f88280b81c0 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 -- 172.21.15.84:0/4216997691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b81c0 msgr2=0x7f88280cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 --2- 172.21.15.84:0/4216997691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b81c0 0x7f88280cf600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8818009870 tx=0x7f8818009b80 comp rx=0 tx=0).stop 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 -- 172.21.15.84:0/4216997691 shutdown_connections 2023-07-06T21:41:08.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 --2- 172.21.15.84:0/4216997691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b81c0 0x7f88280cf600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 -- 172.21.15.84:0/4216997691 >> 172.21.15.84:0/4216997691 conn(0x7f882806b2b0 msgr2=0x7f882806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:08.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 -- 172.21.15.84:0/4216997691 shutdown_connections 2023-07-06T21:41:08.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.468+0000 7f882d7ff700 1 -- 172.21.15.84:0/4216997691 wait complete. 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.470+0000 7f882d7ff700 1 Processor -- start 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.470+0000 7f882d7ff700 1 -- start start 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f882d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b1a80 0x7f8828154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f882d7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 0x7f88280c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f882d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88280ceda0 0x7f88280c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f882d7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280c9cc0 con 0x7f88280b81c0 2023-07-06T21:41:08.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f882d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280c7730 con 0x7f88280ceda0 2023-07-06T21:41:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f882d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88280c7a10 con 0x7f88280b1a80 2023-07-06T21:41:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 0x7f88280c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 0x7f88280c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38738/0 (socket says 172.21.15.84:38738) 2023-07-06T21:41:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 -- 172.21.15.84:0/2011123399 learned_addr learned my addr 172.21.15.84:0/2011123399 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f8826ffd700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b1a80 0x7f8828154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f88277fe700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88280ceda0 0x7f88280c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 -- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b1a80 msgr2=0x7f8828154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b1a80 0x7f8828154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 -- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88280ceda0 msgr2=0x7f88280c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88280ceda0 0x7f88280c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f881ffff700 1 -- 172.21.15.84:0/2011123399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180094a0 con 0x7f88280b81c0 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.471+0000 7f8826ffd700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b1a80 0x7f8828154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:08.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.472+0000 7f881ffff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 0x7f88280c7f10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f881000c520 tx=0x7f881000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.472+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100052d0 con 0x7f88280b81c0 2023-07-06T21:41:08.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.472+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810005430 con 0x7f88280b81c0 2023-07-06T21:41:08.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.472+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88280c9ed0 con 0x7f88280b81c0 2023-07-06T21:41:08.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.472+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88100038c0 con 0x7f88280b81c0 2023-07-06T21:41:08.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.472+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88280c6960 con 0x7f88280b81c0 2023-07-06T21:41:08.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.473+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88280ca3f0 con 0x7f88280b81c0 2023-07-06T21:41:08.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.474+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8810003a20 con 0x7f88280b81c0 2023-07-06T21:41:08.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.475+0000 7f881f7fe700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880806ace0 0x7f880806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.475+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8810088be0 con 0x7f88280b81c0 2023-07-06T21:41:08.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.475+0000 7f8826ffd700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880806ace0 0x7f880806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.476+0000 7f8826ffd700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880806ace0 0x7f880806d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88180090b0 tx=0x7f8818008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.479+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f88280ca3f0 con 0x7f88280b81c0 2023-07-06T21:41:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.673+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f88280ca3f0 con 0x7f88280b81c0 2023-07-06T21:41:08.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.673+0000 7f881f7fe700 1 -- 172.21.15.84:0/2011123399 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7f8810017020 con 0x7f88280b81c0 2023-07-06T21:41:08.676 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880806ace0 msgr2=0x7f880806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880806ace0 0x7f880806d190 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f88180090b0 tx=0x7f8818008030 comp rx=0 tx=0).stop 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 msgr2=0x7f88280c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 0x7f88280c7f10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f881000c520 tx=0x7f881000c8e0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 shutdown_connections 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880806ace0 0x7f880806d190 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88280ceda0 0x7f88280c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280b1a80 0x7f8828154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 --2- 172.21.15.84:0/2011123399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88280b81c0 0x7f88280c7f10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 >> 172.21.15.84:0/2011123399 conn(0x7f882806b2b0 msgr2=0x7f882815af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 shutdown_connections 2023-07-06T21:41:08.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.675+0000 7f882d7ff700 1 -- 172.21.15.84:0/2011123399 wait complete. 2023-07-06T21:41:08.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:41:08.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.887+0000 7fd51cdff700 1 Processor -- start 2023-07-06T21:41:08.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.888+0000 7fd51cdff700 1 -- start start 2023-07-06T21:41:08.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51cdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5180b83c0 0x7fd5180cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5180d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51cdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5180d0a80 con 0x7fd5180d3710 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51cdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5180b8790 con 0x7fd5180b83c0 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180b4e70 con 0x7fd5180cfc60 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5180b83c0 0x7fd5180cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd5180b83c0 0x7fd5180cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38752/0 (socket says 172.21.15.84:38752) 2023-07-06T21:41:08.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 -- 172.21.15.84:0/830395115 learned_addr learned my addr 172.21.15.84:0/830395115 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd516d9d700 1 --2- 172.21.15.84:0/830395115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5180d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 -- 172.21.15.84:0/830395115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 msgr2=0x7fd5180d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 --2- 172.21.15.84:0/830395115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5180d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 -- 172.21.15.84:0/830395115 >> v1:172.21.15.107:6789/0 conn(0x7fd5180d3710 legacy=0x7fd5180cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.889+0000 7fd51659c700 1 -- 172.21.15.84:0/830395115 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5180b5150 con 0x7fd5180b83c0 2023-07-06T21:41:08.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd51659c700 1 --2- 172.21.15.84:0/830395115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180b83c0 0x7fd5180cd200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd5080098f0 tx=0x7fd508009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9f23d4b558c6c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd515d9b700 1 -- 172.21.15.84:0/830395115 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508004230 con 0x7fd5180b83c0 2023-07-06T21:41:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd515d9b700 1 -- 172.21.15.84:0/830395115 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508004390 con 0x7fd5180b83c0 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd515d9b700 1 -- 172.21.15.84:0/830395115 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5080036a0 con 0x7fd5180b83c0 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd51cdff700 1 -- 172.21.15.84:0/830395115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180b83c0 msgr2=0x7fd5180cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd51cdff700 1 --2- 172.21.15.84:0/830395115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180b83c0 0x7fd5180cd200 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd5080098f0 tx=0x7fd508009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd51cdff700 1 -- 172.21.15.84:0/830395115 shutdown_connections 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd51cdff700 1 --2- 172.21.15.84:0/830395115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5180d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.890+0000 7fd51cdff700 1 --2- 172.21.15.84:0/830395115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180b83c0 0x7fd5180cd200 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.891+0000 7fd51cdff700 1 -- 172.21.15.84:0/830395115 >> 172.21.15.84:0/830395115 conn(0x7fd51806b350 msgr2=0x7fd51806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:08.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.891+0000 7fd51cdff700 1 -- 172.21.15.84:0/830395115 shutdown_connections 2023-07-06T21:41:08.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.891+0000 7fd51cdff700 1 -- 172.21.15.84:0/830395115 wait complete. 2023-07-06T21:41:08.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.892+0000 7fd51cdff700 1 Processor -- start 2023-07-06T21:41:08.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.892+0000 7fd51cdff700 1 -- start start 2023-07-06T21:41:08.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd51cdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5180b83c0 0x7fd518150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd51cdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5181592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd51cdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 0x7fd518159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd51cdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180cc8a0 con 0x7fd5180d3710 2023-07-06T21:41:08.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd51cdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180cbc20 con 0x7fd5180b83c0 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd51cdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5180cbf00 con 0x7fd5180cfc60 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.893+0000 7fd516d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 0x7fd518159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 0x7fd518159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38764/0 (socket says 172.21.15.84:38764) 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 -- 172.21.15.84:0/3951975853 learned_addr learned my addr 172.21.15.84:0/3951975853 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd51659c700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5180b83c0 0x7fd518150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd50ffff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5181592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 -- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 msgr2=0x7fd5181592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5181592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 -- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5180b83c0 msgr2=0x7fd518150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5180b83c0 0x7fd518150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 -- 172.21.15.84:0/3951975853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd508009550 con 0x7fd5180d3710 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd516d9d700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 0x7fd518159800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd510009fd0 tx=0x7fd51000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd51000ede0 con 0x7fd5180d3710 2023-07-06T21:41:08.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510014420 con 0x7fd5180d3710 2023-07-06T21:41:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5180cc1e0 con 0x7fd5180d3710 2023-07-06T21:41:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.894+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100147d0 con 0x7fd5180d3710 2023-07-06T21:41:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.895+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd518151500 con 0x7fd5180d3710 2023-07-06T21:41:08.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.895+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5180cc9e0 con 0x7fd5180d3710 2023-07-06T21:41:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.896+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd510014930 con 0x7fd5180d3710 2023-07-06T21:41:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.897+0000 7fd50effd700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4f806ace0 0x7fd4f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.897+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd510089dc0 con 0x7fd5180d3710 2023-07-06T21:41:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.898+0000 7fd51659c700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4f806ace0 0x7fd4f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:08.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.898+0000 7fd51659c700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4f806ace0 0x7fd4f806d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd508009130 tx=0x7fd508008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:08.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:08.901+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd510054ce0 con 0x7fd5180d3710 2023-07-06T21:41:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.119+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd5180d0fb0 con 0x7fd5180d3710 2023-07-06T21:41:09.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.119+0000 7fd50effd700 1 -- 172.21.15.84:0/3951975853 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd510058300 con 0x7fd5180d3710 2023-07-06T21:41:09.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4f806ace0 msgr2=0x7fd4f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4f806ace0 0x7fd4f806d190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd508009130 tx=0x7fd508008030 comp rx=0 tx=0).stop 2023-07-06T21:41:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 msgr2=0x7fd518159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 0x7fd518159800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd510009fd0 tx=0x7fd51000e3c0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 shutdown_connections 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4f806ace0 0x7fd4f806d190 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd5180d3710 0x7fd518159800 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5180b83c0 0x7fd518150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 --2- 172.21.15.84:0/3951975853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5180cfc60 0x7fd5181592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.121+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 >> 172.21.15.84:0/3951975853 conn(0x7fd51806b350 msgr2=0x7fd51815a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.122+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 shutdown_connections 2023-07-06T21:41:09.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.122+0000 7fd51cdff700 1 -- 172.21.15.84:0/3951975853 wait complete. 2023-07-06T21:41:09.127 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:41:09.142 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:09.142 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:09.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:09.417 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.415+0000 7fdb0e7ff700 1 Processor -- start 2023-07-06T21:41:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.415+0000 7fdb0e7ff700 1 -- start start 2023-07-06T21:41:09.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb0e7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb0815a7e0 0x7fdb0815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb0e7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdb080c8d00 0x7fdb080c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb0e7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb08070230 con 0x7fdb08154910 2023-07-06T21:41:09.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb0e7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb08071390 con 0x7fdb0815a7e0 2023-07-06T21:41:09.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb0e7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080c9660 con 0x7fdb080c8d00 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb077fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb0815a7e0 0x7fdb0815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb077fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb0815a7e0 0x7fdb0815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38768/0 (socket says 172.21.15.84:38768) 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb077fe700 1 -- 172.21.15.84:0/3595158349 learned_addr learned my addr 172.21.15.84:0/3595158349 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb0cd9c700 1 --2- 172.21.15.84:0/3595158349 >> v2:172.21.15.107:3300/0 conn(0x7fdb080c8d00 0x7fdb080c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb077fe700 1 -- 172.21.15.84:0/3595158349 >> v2:172.21.15.107:3300/0 conn(0x7fdb080c8d00 msgr2=0x7fdb080c9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb077fe700 1 --2- 172.21.15.84:0/3595158349 >> v2:172.21.15.107:3300/0 conn(0x7fdb080c8d00 0x7fdb080c9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.417+0000 7fdb077fe700 1 -- 172.21.15.84:0/3595158349 >> v1:172.21.15.107:6789/0 conn(0x7fdb08154910 legacy=0x7fdb08070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb077fe700 1 -- 172.21.15.84:0/3595158349 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0815b2a0 con 0x7fdb0815a7e0 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb0cd9c700 1 --2- 172.21.15.84:0/3595158349 >> v2:172.21.15.107:3300/0 conn(0x7fdb080c8d00 0x7fdb080c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb077fe700 1 --2- 172.21.15.84:0/3595158349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb0815ac30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdb0806b020 tx=0x7fdafc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad85929182c7d49 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb06ffd700 1 -- 172.21.15.84:0/3595158349 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc004230 con 0x7fdb0815a7e0 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb06ffd700 1 -- 172.21.15.84:0/3595158349 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc004390 con 0x7fdb0815a7e0 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb06ffd700 1 -- 172.21.15.84:0/3595158349 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0036a0 con 0x7fdb0815a7e0 2023-07-06T21:41:09.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3595158349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 msgr2=0x7fdb0815ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.418+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3595158349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb0815ac30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdb0806b020 tx=0x7fdafc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.419+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3595158349 shutdown_connections 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.419+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3595158349 >> v2:172.21.15.107:3300/0 conn(0x7fdb080c8d00 0x7fdb080c9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.419+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3595158349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb0815ac30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.419+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3595158349 >> 172.21.15.84:0/3595158349 conn(0x7fdb080cf3d0 msgr2=0x7fdb080cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.419+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3595158349 shutdown_connections 2023-07-06T21:41:09.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.419+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3595158349 wait complete. 2023-07-06T21:41:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.420+0000 7fdb0e7ff700 1 Processor -- start 2023-07-06T21:41:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.420+0000 7fdb0e7ff700 1 -- start start 2023-07-06T21:41:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.421+0000 7fdb0e7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb080c8d00 0x7fdb080b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.421+0000 7fdb0e7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb08154910 0x7fdb0806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.421+0000 7fdb0e7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb08137280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.421+0000 7fdb0e7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0806c8e0 con 0x7fdb0815a7e0 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.421+0000 7fdb0e7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0806cb80 con 0x7fdb08154910 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.421+0000 7fdb0e7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0806ce60 con 0x7fdb080c8d00 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb08137280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb07fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb080c8d00 0x7fdb080b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb08137280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38774/0 (socket says 172.21.15.84:38774) 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 -- 172.21.15.84:0/3855990977 learned_addr learned my addr 172.21.15.84:0/3855990977 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:09.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb077fe700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb08154910 0x7fdb0806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 -- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb080c8d00 msgr2=0x7fdb080b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb080c8d00 0x7fdb080b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 -- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb08154910 msgr2=0x7fdb0806c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb08154910 0x7fdb0806c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 -- 172.21.15.84:0/3855990977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009550 con 0x7fdb0815a7e0 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.422+0000 7fdb0cd9c700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb08137280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000cef0 tx=0x7fdb0000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:09.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.423+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000ad00 con 0x7fdb0815a7e0 2023-07-06T21:41:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.423+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0000ae60 con 0x7fdb0815a7e0 2023-07-06T21:41:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.423+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0806d140 con 0x7fdb0815a7e0 2023-07-06T21:41:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.423+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00015680 con 0x7fdb0815a7e0 2023-07-06T21:41:09.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.423+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0806d700 con 0x7fdb0815a7e0 2023-07-06T21:41:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.424+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0806d920 con 0x7fdb0815a7e0 2023-07-06T21:41:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.425+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdb0000d430 con 0x7fdb0815a7e0 2023-07-06T21:41:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.426+0000 7fdb04ff9700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdaf406ace0 0x7fdaf406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.426+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdb00089b60 con 0x7fdb0815a7e0 2023-07-06T21:41:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.426+0000 7fdb07fff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdaf406ace0 0x7fdaf406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.426+0000 7fdb07fff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdaf406ace0 0x7fdaf406d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8007fd0 tx=0x7fdaf8007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:09.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.430+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdb000549e0 con 0x7fdb0815a7e0 2023-07-06T21:41:09.630 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.629+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb0804e240 con 0x7fdb0815a7e0 2023-07-06T21:41:09.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.630+0000 7fdb04ff9700 1 -- 172.21.15.84:0/3855990977 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7fdb0001a090 con 0x7fdb0815a7e0 2023-07-06T21:41:09.632 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:09.633 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:09.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.632+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdaf406ace0 msgr2=0x7fdaf406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.632+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdaf406ace0 0x7fdaf406d190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8007fd0 tx=0x7fdaf8007ec0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.632+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 msgr2=0x7fdb08137280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.632+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb08137280 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000cef0 tx=0x7fdb0000cf20 comp rx=0 tx=0).stop 2023-07-06T21:41:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 shutdown_connections 2023-07-06T21:41:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdaf406ace0 0x7fdaf406d190 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb0815a7e0 0x7fdb08137280 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb080c8d00 0x7fdb080b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 --2- 172.21.15.84:0/3855990977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb08154910 0x7fdb0806c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 >> 172.21.15.84:0/3855990977 conn(0x7fdb080cf3d0 msgr2=0x7fdb080c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:09.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 shutdown_connections 2023-07-06T21:41:09.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.633+0000 7fdb0e7ff700 1 -- 172.21.15.84:0/3855990977 wait complete. 2023-07-06T21:41:09.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:09.654 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:09.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.851+0000 7f5b099ff700 1 Processor -- start 2023-07-06T21:41:09.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.851+0000 7f5b099ff700 1 -- start start 2023-07-06T21:41:09.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.852+0000 7f5b099ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b040b81c0 0x7f5b040cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.852+0000 7f5b099ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b040d1830 con 0x7f5b040b1a80 2023-07-06T21:41:09.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.852+0000 7f5b099ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b04160d40 con 0x7f5b040ceda0 2023-07-06T21:41:09.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.852+0000 7f5b099ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b040cf330 con 0x7f5b040b81c0 2023-07-06T21:41:09.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5afbfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5b040b1a80 0x7f5b0415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54016/0 (socket says 172.21.15.84:54016) 2023-07-06T21:41:09.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5afbfff700 1 -- 172.21.15.84:0/3701613871 learned_addr learned my addr 172.21.15.84:0/3701613871 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:09.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5b02ffd700 1 --2- 172.21.15.84:0/3701613871 >> v2:172.21.15.84:3300/0 conn(0x7f5b040b81c0 0x7f5b040cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5b027fc700 1 -- 172.21.15.84:0/3701613871 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636639127 0 0) 0x7f5b040d1830 con 0x7f5b040b1a80 2023-07-06T21:41:09.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5b027fc700 1 -- 172.21.15.84:0/3701613871 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af00035b0 con 0x7f5b040b1a80 2023-07-06T21:41:09.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3701613871 >> v1:172.21.15.107:6789/0 conn(0x7f5b040ceda0 legacy=0x7f5b04160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3701613871 >> v1:172.21.15.84:6789/0 conn(0x7f5b040b1a80 legacy=0x7f5b0415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.853+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3701613871 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040d1270 con 0x7f5b040b81c0 2023-07-06T21:41:09.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b02ffd700 1 --2- 172.21.15.84:0/3701613871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040b81c0 0x7f5b040cf600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009840 tx=0x7f5af4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebbba65d2a97808 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b027fc700 1 -- 172.21.15.84:0/3701613871 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4011830 con 0x7f5b040b81c0 2023-07-06T21:41:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b027fc700 1 -- 172.21.15.84:0/3701613871 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af4011990 con 0x7f5b040b81c0 2023-07-06T21:41:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b027fc700 1 -- 172.21.15.84:0/3701613871 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4011d20 con 0x7f5b040b81c0 2023-07-06T21:41:09.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 -- 172.21.15.84:0/3701613871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040b81c0 msgr2=0x7f5b040cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 --2- 172.21.15.84:0/3701613871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040b81c0 0x7f5b040cf600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009840 tx=0x7f5af4009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 -- 172.21.15.84:0/3701613871 shutdown_connections 2023-07-06T21:41:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 --2- 172.21.15.84:0/3701613871 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040b81c0 0x7f5b040cf600 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 -- 172.21.15.84:0/3701613871 >> 172.21.15.84:0/3701613871 conn(0x7f5b0406b2b0 msgr2=0x7f5b0406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 -- 172.21.15.84:0/3701613871 shutdown_connections 2023-07-06T21:41:09.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.854+0000 7f5b099ff700 1 -- 172.21.15.84:0/3701613871 wait complete. 2023-07-06T21:41:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.856+0000 7f5b099ff700 1 Processor -- start 2023-07-06T21:41:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.856+0000 7f5b099ff700 1 -- start start 2023-07-06T21:41:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b099ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b040b1a80 0x7f5b04158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b099ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b040b81c0 0x7f5b04160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b099ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 0x7f5b041e5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b099ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041e6900 con 0x7f5b040ceda0 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b099ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041e6b50 con 0x7f5b040b81c0 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b099ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b041e6e30 con 0x7f5b040b1a80 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b037fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 0x7f5b041e5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b037fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 0x7f5b041e5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38788/0 (socket says 172.21.15.84:38788) 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.857+0000 7f5b037fe700 1 -- 172.21.15.84:0/273478870 learned_addr learned my addr 172.21.15.84:0/273478870 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5afbfff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b040b81c0 0x7f5b04160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b02ffd700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b040b1a80 0x7f5b04158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b037fe700 1 -- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b040b1a80 msgr2=0x7f5b04158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b037fe700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b040b1a80 0x7f5b04158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b037fe700 1 -- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b040b81c0 msgr2=0x7f5b04160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b037fe700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b040b81c0 0x7f5b04160390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b037fe700 1 -- 172.21.15.84:0/273478870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40094a0 con 0x7f5b040ceda0 2023-07-06T21:41:09.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b037fe700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 0x7f5b041e5c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00cb50 tx=0x7f5afc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc003c00 con 0x7f5b040ceda0 2023-07-06T21:41:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc003d60 con 0x7f5b040ceda0 2023-07-06T21:41:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b041e7110 con 0x7f5b040ceda0 2023-07-06T21:41:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00aa20 con 0x7f5b040ceda0 2023-07-06T21:41:09.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.858+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b040cb0c0 con 0x7f5b040ceda0 2023-07-06T21:41:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.859+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0404e1d0 con 0x7f5b040ceda0 2023-07-06T21:41:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.860+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5afc00ab80 con 0x7f5b040ceda0 2023-07-06T21:41:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.861+0000 7f5b00ff9700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5ae406ace0 0x7f5ae406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:09.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.861+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5afc089630 con 0x7f5b040ceda0 2023-07-06T21:41:09.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.862+0000 7f5b02ffd700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5ae406ace0 0x7f5ae406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.862+0000 7f5b02ffd700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5ae406ace0 0x7f5ae406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5af40090b0 tx=0x7f5af4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:09.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:09.866+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5afc054540 con 0x7f5b040ceda0 2023-07-06T21:41:10.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.069+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b0415a690 con 0x7f5b040ceda0 2023-07-06T21:41:10.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.069+0000 7f5b00ff9700 1 -- 172.21.15.84:0/273478870 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5afc057b60 con 0x7f5b040ceda0 2023-07-06T21:41:10.072 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5ae406ace0 msgr2=0x7f5ae406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5ae406ace0 0x7f5ae406d190 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5af40090b0 tx=0x7f5af4008030 comp rx=0 tx=0).stop 2023-07-06T21:41:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 msgr2=0x7f5b041e5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 0x7f5b041e5c40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5afc00cb50 tx=0x7f5afc00ce60 comp rx=0 tx=0).stop 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 shutdown_connections 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5ae406ace0 0x7f5ae406d190 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b040ceda0 0x7f5b041e5c40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b040b1a80 0x7f5b04158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 --2- 172.21.15.84:0/273478870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b040b81c0 0x7f5b04160390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 >> 172.21.15.84:0/273478870 conn(0x7f5b0406b2b0 msgr2=0x7f5b040d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:10.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 shutdown_connections 2023-07-06T21:41:10.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.072+0000 7f5b099ff700 1 -- 172.21.15.84:0/273478870 wait complete. 2023-07-06T21:41:10.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:10.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.289+0000 7ff65a5ff700 1 Processor -- start 2023-07-06T21:41:10.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.289+0000 7ff65a5ff700 1 -- start start 2023-07-06T21:41:10.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff65a5ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff654069e90 0x7ff65406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff65a5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff65406dde0 0x7ff65406a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff65a5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6540cf280 0x7ff6540b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff65a5ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6540c94b0 con 0x7ff654069e90 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff65a5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff65406d350 con 0x7ff65406dde0 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff65a5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65406a7a0 con 0x7ff6540cf280 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff653fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff654069e90 0x7ff65406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff653fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7ff654069e90 0x7ff65406ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38802/0 (socket says 172.21.15.84:38802) 2023-07-06T21:41:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.290+0000 7ff653fff700 1 -- 172.21.15.84:0/1381142732 learned_addr learned my addr 172.21.15.84:0/1381142732 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff658b9c700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6540cf280 0x7ff6540b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff6537fe700 1 --2- 172.21.15.84:0/1381142732 >> v2:172.21.15.107:3300/0 conn(0x7ff65406dde0 0x7ff65406a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff653fff700 1 -- 172.21.15.84:0/1381142732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6540cf280 msgr2=0x7ff6540b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:10.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff653fff700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6540cf280 0x7ff6540b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff653fff700 1 -- 172.21.15.84:0/1381142732 >> v2:172.21.15.107:3300/0 conn(0x7ff65406dde0 msgr2=0x7ff65406a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:10.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff653fff700 1 --2- 172.21.15.84:0/1381142732 >> v2:172.21.15.107:3300/0 conn(0x7ff65406dde0 0x7ff65406a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff653fff700 1 -- 172.21.15.84:0/1381142732 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6540ca9b0 con 0x7ff654069e90 2023-07-06T21:41:10.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff6537fe700 1 --2- 172.21.15.84:0/1381142732 >> v2:172.21.15.107:3300/0 conn(0x7ff65406dde0 0x7ff65406a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:10.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff658b9c700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6540cf280 0x7ff6540b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:10.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff653fff700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff65406ce10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff65406ac00 tx=0x7ff644009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4314fa47a0243a93 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:10.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff652ffd700 1 -- 172.21.15.84:0/1381142732 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400b870 con 0x7ff654069e90 2023-07-06T21:41:10.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff652ffd700 1 -- 172.21.15.84:0/1381142732 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64400b9d0 con 0x7ff654069e90 2023-07-06T21:41:10.299 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.291+0000 7ff652ffd700 1 -- 172.21.15.84:0/1381142732 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400bd60 con 0x7ff654069e90 2023-07-06T21:41:10.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1381142732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 msgr2=0x7ff65406ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff65406ce10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff65406ac00 tx=0x7ff644009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1381142732 shutdown_connections 2023-07-06T21:41:10.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6540cf280 0x7ff6540b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1381142732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff65406ce10 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1381142732 >> v2:172.21.15.107:3300/0 conn(0x7ff65406dde0 0x7ff65406a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1381142732 >> 172.21.15.84:0/1381142732 conn(0x7ff6540cbf10 msgr2=0x7ff6540ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:10.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1381142732 shutdown_connections 2023-07-06T21:41:10.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.292+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1381142732 wait complete. 2023-07-06T21:41:10.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.294+0000 7ff65a5ff700 1 Processor -- start 2023-07-06T21:41:10.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.294+0000 7ff65a5ff700 1 -- start start 2023-07-06T21:41:10.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff65a5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff6541f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff65a5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff65406dde0 0x7ff654150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff65a5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6540cf280 0x7ff6540b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff65a5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65415a950 con 0x7ff654069e90 2023-07-06T21:41:10.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff65a5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65415af10 con 0x7ff6540cf280 2023-07-06T21:41:10.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff65a5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6541f4600 con 0x7ff65406dde0 2023-07-06T21:41:10.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff653fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff6541f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff653fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff6541f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38812/0 (socket says 172.21.15.84:38812) 2023-07-06T21:41:10.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff653fff700 1 -- 172.21.15.84:0/1500081513 learned_addr learned my addr 172.21.15.84:0/1500081513 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:10.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff658b9c700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6540cf280 0x7ff6540b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.295+0000 7ff6537fe700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff65406dde0 0x7ff654150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff653fff700 1 -- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff65406dde0 msgr2=0x7ff654150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff653fff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff65406dde0 0x7ff654150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff653fff700 1 -- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6540cf280 msgr2=0x7ff6540b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff653fff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6540cf280 0x7ff6540b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff653fff700 1 -- 172.21.15.84:0/1500081513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff644009550 con 0x7ff654069e90 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff653fff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff6541f40c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff654071ce0 tx=0x7ff644004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644004520 con 0x7ff654069e90 2023-07-06T21:41:10.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64401e070 con 0x7ff654069e90 2023-07-06T21:41:10.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6440038c0 con 0x7ff654069e90 2023-07-06T21:41:10.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6540c4420 con 0x7ff654069e90 2023-07-06T21:41:10.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.296+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff65415aba0 con 0x7ff654069e90 2023-07-06T21:41:10.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.297+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff65415ad30 con 0x7ff654069e90 2023-07-06T21:41:10.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.298+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff644003a20 con 0x7ff654069e90 2023-07-06T21:41:10.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.299+0000 7ff650ff9700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff64006ace0 0x7ff64006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.299+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff644020070 con 0x7ff654069e90 2023-07-06T21:41:10.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.299+0000 7ff6537fe700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff64006ace0 0x7ff64006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.300+0000 7ff6537fe700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff64006ace0 0x7ff64006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6540b3330 tx=0x7ff648008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:10.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.303+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff65415ad30 con 0x7ff654069e90 2023-07-06T21:41:10.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.497+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff65415ad30 con 0x7ff654069e90 2023-07-06T21:41:10.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.497+0000 7ff650ff9700 1 -- 172.21.15.84:0/1500081513 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff644055a90 con 0x7ff654069e90 2023-07-06T21:41:10.500 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:10.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff64006ace0 msgr2=0x7ff64006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff64006ace0 0x7ff64006d190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff6540b3330 tx=0x7ff648008040 comp rx=0 tx=0).stop 2023-07-06T21:41:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 msgr2=0x7ff6541f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff6541f40c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff654071ce0 tx=0x7ff644004750 comp rx=0 tx=0).stop 2023-07-06T21:41:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 shutdown_connections 2023-07-06T21:41:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff64006ace0 0x7ff64006d190 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff6540cf280 0x7ff6540b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff654069e90 0x7ff6541f40c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.499+0000 7ff65a5ff700 1 --2- 172.21.15.84:0/1500081513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff65406dde0 0x7ff654150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.500+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 >> 172.21.15.84:0/1500081513 conn(0x7ff6540cbf10 msgr2=0x7ff6540ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.500+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 shutdown_connections 2023-07-06T21:41:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.500+0000 7ff65a5ff700 1 -- 172.21.15.84:0/1500081513 wait complete. 2023-07-06T21:41:10.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:10.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.714+0000 7f8826dff700 1 Processor -- start 2023-07-06T21:41:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.714+0000 7f8826dff700 1 -- start start 2023-07-06T21:41:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.715+0000 7f8826dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b81c0 0x7f88200cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.715+0000 7f8826dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 0x7f88200cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.715+0000 7f8826dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88200d1830 con 0x7f88200ceda0 2023-07-06T21:41:10.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.715+0000 7f8826dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88200cf330 con 0x7f88200b1a80 2023-07-06T21:41:10.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.715+0000 7f8826dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88200d1270 con 0x7f88200b81c0 2023-07-06T21:41:10.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f881ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 0x7f88200cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f8824b9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b81c0 0x7f88200cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f882539c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f88200ceda0 0x7f882015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:59476/0 (socket says 172.21.15.84:59476) 2023-07-06T21:41:10.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f881ffff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 0x7f88200cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:33382/0 (socket says 172.21.15.84:33382) 2023-07-06T21:41:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f882539c700 1 -- 172.21.15.84:0/736563860 learned_addr learned my addr 172.21.15.84:0/736563860 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f881f7fe700 1 -- 172.21.15.84:0/736563860 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473389201 0 0) 0x7f88200d1830 con 0x7f88200ceda0 2023-07-06T21:41:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f881f7fe700 1 -- 172.21.15.84:0/736563860 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88100035b0 con 0x7f88200ceda0 2023-07-06T21:41:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f8824b9b700 1 -- 172.21.15.84:0/736563860 >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 msgr2=0x7f88200cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f8824b9b700 1 --2- 172.21.15.84:0/736563860 >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 0x7f88200cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f8824b9b700 1 -- 172.21.15.84:0/736563860 >> v1:172.21.15.107:6789/0 conn(0x7f88200ceda0 legacy=0x7f882015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f8824b9b700 1 -- 172.21.15.84:0/736563860 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200d1550 con 0x7f88200b81c0 2023-07-06T21:41:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.716+0000 7f881ffff700 1 --2- 172.21.15.84:0/736563860 >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 0x7f88200cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:10.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8824b9b700 1 --2- 172.21.15.84:0/736563860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b81c0 0x7f88200cf600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8814009840 tx=0x7f8814009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18807325d00e15c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f881f7fe700 1 -- 172.21.15.84:0/736563860 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814011830 con 0x7f88200b81c0 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f881f7fe700 1 -- 172.21.15.84:0/736563860 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8814011990 con 0x7f88200b81c0 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f881f7fe700 1 -- 172.21.15.84:0/736563860 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814011d20 con 0x7f88200b81c0 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 -- 172.21.15.84:0/736563860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b81c0 msgr2=0x7f88200cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 --2- 172.21.15.84:0/736563860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b81c0 0x7f88200cf600 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8814009840 tx=0x7f8814009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 -- 172.21.15.84:0/736563860 shutdown_connections 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 --2- 172.21.15.84:0/736563860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b81c0 0x7f88200cf600 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 --2- 172.21.15.84:0/736563860 >> v2:172.21.15.107:3300/0 conn(0x7f88200b1a80 0x7f88200cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 -- 172.21.15.84:0/736563860 >> 172.21.15.84:0/736563860 conn(0x7f882006b2b0 msgr2=0x7f882006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.717+0000 7f8826dff700 1 -- 172.21.15.84:0/736563860 shutdown_connections 2023-07-06T21:41:10.728 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.718+0000 7f8826dff700 1 -- 172.21.15.84:0/736563860 wait complete. 2023-07-06T21:41:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.719+0000 7f8826dff700 1 Processor -- start 2023-07-06T21:41:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.719+0000 7f8826dff700 1 -- start start 2023-07-06T21:41:10.731 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.720+0000 7f8826dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b1a80 0x7f8820158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.720+0000 7f8826dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88200b81c0 0x7f88201e6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.720+0000 7f8826dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 0x7f88201e29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.720+0000 7f8826dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88200c6720 con 0x7f88200ceda0 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.720+0000 7f8826dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88201e6e60 con 0x7f88200b81c0 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.720+0000 7f8826dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8820158750 con 0x7f88200b1a80 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 0x7f88201e29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.732 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 0x7f88201e29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38828/0 (socket says 172.21.15.84:38828) 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 -- 172.21.15.84:0/2569387145 learned_addr learned my addr 172.21.15.84:0/2569387145 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f881ffff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88200b81c0 0x7f88201e6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f8824b9b700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b1a80 0x7f8820158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 -- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b1a80 msgr2=0x7f8820158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b1a80 0x7f8820158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 -- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88200b81c0 msgr2=0x7f88201e6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88200b81c0 0x7f88201e6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.733 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 -- 172.21.15.84:0/2569387145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0094a0 con 0x7f88200ceda0 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f881ffff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88200b81c0 0x7f88201e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.721+0000 7f882539c700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 0x7f88201e29f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f881800caa0 tx=0x7f881800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.722+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800aa80 con 0x7f88200ceda0 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.722+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881800abe0 con 0x7f88200ceda0 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.722+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818012400 con 0x7f88200ceda0 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.722+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88140094a0 con 0x7f88200ceda0 2023-07-06T21:41:10.734 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.722+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88200c6c10 con 0x7f88200ceda0 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.723+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882004e1d0 con 0x7f88200ceda0 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.724+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f881801b3b0 con 0x7f88200ceda0 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.725+0000 7f881d7fa700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880406ace0 0x7f880406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.725+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8818089360 con 0x7f88200ceda0 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.725+0000 7f8824b9b700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880406ace0 0x7f880406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.726+0000 7f8824b9b700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880406ace0 0x7f880406d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8814000c00 tx=0x7f8814008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:10.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.729+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8818054270 con 0x7f88200ceda0 2023-07-06T21:41:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.927+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88200cfd00 con 0x7f88200ceda0 2023-07-06T21:41:10.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.927+0000 7f881d7fa700 1 -- 172.21.15.84:0/2569387145 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8818057890 con 0x7f88200ceda0 2023-07-06T21:41:10.930 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:10.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880406ace0 msgr2=0x7f880406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880406ace0 0x7f880406d190 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8814000c00 tx=0x7f8814008030 comp rx=0 tx=0).stop 2023-07-06T21:41:10.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 msgr2=0x7f88201e29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:10.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 0x7f88201e29f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f881800caa0 tx=0x7f881800ce60 comp rx=0 tx=0).stop 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 shutdown_connections 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f880406ace0 0x7f880406d190 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88200ceda0 0x7f88201e29f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88200b1a80 0x7f8820158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 --2- 172.21.15.84:0/2569387145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88200b81c0 0x7f88201e6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 >> 172.21.15.84:0/2569387145 conn(0x7f882006b2b0 msgr2=0x7f88200b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:10.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 shutdown_connections 2023-07-06T21:41:10.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:10.930+0000 7f8826dff700 1 -- 172.21.15.84:0/2569387145 wait complete. 2023-07-06T21:41:10.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.144+0000 7f88b23ff700 1 Processor -- start 2023-07-06T21:41:11.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.144+0000 7f88b23ff700 1 -- start start 2023-07-06T21:41:11.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.145+0000 7f88b23ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f88ac0b81c0 0x7f88ac0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.145+0000 7f88b23ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac0d1830 con 0x7f88ac0ceda0 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.145+0000 7f88b23ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ac160d40 con 0x7f88ac0b1a80 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.145+0000 7f88b23ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88ac0cf330 con 0x7f88ac0b81c0 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.145+0000 7f88b099c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f88ac0ceda0 0x7f88ac160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:54026/0 (socket says 172.21.15.84:54026) 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.145+0000 7f88b099c700 1 -- 172.21.15.84:0/2610134327 learned_addr learned my addr 172.21.15.84:0/2610134327 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773036862 0 0) 0x7f88ac0d1830 con 0x7f88ac0ceda0 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88abfff700 1 --2- 172.21.15.84:0/2610134327 >> v2:172.21.15.107:3300/0 conn(0x7f88ac0b81c0 0x7f88ac0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88940035b0 con 0x7f88ac0ceda0 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637517133 0 0) 0x7f88940035b0 con 0x7f88ac0ceda0 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 >> v2:172.21.15.107:3300/0 conn(0x7f88ac0b81c0 msgr2=0x7f88ac0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 --2- 172.21.15.84:0/2610134327 >> v2:172.21.15.107:3300/0 conn(0x7f88ac0b81c0 0x7f88ac0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 >> v1:172.21.15.107:6789/0 conn(0x7f88ac0b1a80 legacy=0x7f88ac15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0d1270 con 0x7f88ac0ceda0 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f88a40019f0 con 0x7f88ac0ceda0 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 >> v1:172.21.15.84:6789/0 conn(0x7f88ac0ceda0 legacy=0x7f88ac160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8894003a00 0x7f8894003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8894004850 0x7f8894004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894005200 0x7f889400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0cf330 con 0x7f8894004850 2023-07-06T21:41:11.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889400c080 con 0x7f8894005200 2023-07-06T21:41:11.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889400c240 con 0x7f8894003a00 2023-07-06T21:41:11.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.146+0000 7f88ab7fe700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8894004850 0x7f8894004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88abfff700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8894003a00 0x7f8894003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88b099c700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894005200 0x7f889400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88ab7fe700 1 -- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8894003a00 msgr2=0x7f8894003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88ab7fe700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8894003a00 0x7f8894003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88ab7fe700 1 -- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894005200 msgr2=0x7f889400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88ab7fe700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894005200 0x7f889400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88ab7fe700 1 -- 172.21.15.84:0/2610134327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940035b0 con 0x7f8894004850 2023-07-06T21:41:11.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88b099c700 1 -- 172.21.15.84:0/2610134327 reap_dead start 2023-07-06T21:41:11.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88ab7fe700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8894004850 0x7f8894004cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f88a00017c0 tx=0x7f8894005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db68aac1fbc978 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:11.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00052d0 con 0x7f8894004850 2023-07-06T21:41:11.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0005450 con 0x7f8894004850 2023-07-06T21:41:11.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.147+0000 7f88aaffd700 1 -- 172.21.15.84:0/2610134327 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00038c0 con 0x7f8894004850 2023-07-06T21:41:11.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 -- 172.21.15.84:0/2610134327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8894004850 msgr2=0x7f8894004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8894004850 0x7f8894004cc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f88a00017c0 tx=0x7f8894005880 comp rx=0 tx=0).stop 2023-07-06T21:41:11.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 -- 172.21.15.84:0/2610134327 shutdown_connections 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 --2- 172.21.15.84:0/2610134327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8894004850 0x7f8894004cc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 -- 172.21.15.84:0/2610134327 >> 172.21.15.84:0/2610134327 conn(0x7f88ac06b2b0 msgr2=0x7f88ac06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 -- 172.21.15.84:0/2610134327 shutdown_connections 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.148+0000 7f88b23ff700 1 -- 172.21.15.84:0/2610134327 wait complete. 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.150+0000 7f88b23ff700 1 Processor -- start 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.150+0000 7f88b23ff700 1 -- start start 2023-07-06T21:41:11.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b23ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894004850 0x7f88ac150c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b23ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88ac0c7410 0x7f88ac0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b23ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 0x7f88ac1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b23ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac1563c0 con 0x7f88ac0c6c60 2023-07-06T21:41:11.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b23ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0c7100 con 0x7f8894004850 2023-07-06T21:41:11.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b23ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0d34d0 con 0x7f88ac0c7410 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b099c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 0x7f88ac1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b099c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 0x7f88ac1567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38846/0 (socket says 172.21.15.84:38846) 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88b099c700 1 -- 172.21.15.84:0/1299070974 learned_addr learned my addr 172.21.15.84:0/1299070974 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88abfff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894004850 0x7f88ac150c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.151+0000 7f88ab7fe700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88ac0c7410 0x7f88ac0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b099c700 1 -- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88ac0c7410 msgr2=0x7f88ac0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b099c700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88ac0c7410 0x7f88ac0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b099c700 1 -- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894004850 msgr2=0x7f88ac150c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b099c700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894004850 0x7f88ac150c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b099c700 1 -- 172.21.15.84:0/1299070974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0c7880 con 0x7f88ac0c6c60 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b099c700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 0x7f88ac1567e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f88ac067270 tx=0x7f88a400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4001ec0 con 0x7f88ac0c6c60 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a400ad10 con 0x7f88ac0c6c60 2023-07-06T21:41:11.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac0d31f0 con 0x7f88ac0c6c60 2023-07-06T21:41:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4013630 con 0x7f88ac0c6c60 2023-07-06T21:41:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.152+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac155fe0 con 0x7f88ac0c6c60 2023-07-06T21:41:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.153+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac156500 con 0x7f88ac0c6c60 2023-07-06T21:41:11.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.154+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f88a4013790 con 0x7f88ac0c6c60 2023-07-06T21:41:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.155+0000 7f88a8ff9700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f889806ad10 0x7f889806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.155+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f88a4089ec0 con 0x7f88ac0c6c60 2023-07-06T21:41:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.156+0000 7f88abfff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f889806ad10 0x7f889806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.156+0000 7f88abfff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f889806ad10 0x7f889806d1c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88ac068f30 tx=0x7f889c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:11.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.160+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f88ac156500 con 0x7f88ac0c6c60 2023-07-06T21:41:11.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.352+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88ac156500 con 0x7f88ac0c6c60 2023-07-06T21:41:11.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.352+0000 7f88a8ff9700 1 -- 172.21.15.84:0/1299070974 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f88ac156500 con 0x7f88ac0c6c60 2023-07-06T21:41:11.355 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.354+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f889806ad10 msgr2=0x7f889806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.354+0000 7f88b23ff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f889806ad10 0x7f889806d1c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88ac068f30 tx=0x7f889c009210 comp rx=0 tx=0).stop 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.354+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 msgr2=0x7f88ac1567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.354+0000 7f88b23ff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 0x7f88ac1567e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f88ac067270 tx=0x7f88a400a5f0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 shutdown_connections 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f889806ad10 0x7f889806d1c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88ac0c6c60 0x7f88ac1567e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8894004850 0x7f88ac150c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 --2- 172.21.15.84:0/1299070974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88ac0c7410 0x7f88ac0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 >> 172.21.15.84:0/1299070974 conn(0x7f88ac06b2b0 msgr2=0x7f88ac15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:11.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 shutdown_connections 2023-07-06T21:41:11.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.355+0000 7f88b23ff700 1 -- 172.21.15.84:0/1299070974 wait complete. 2023-07-06T21:41:11.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:11.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.570+0000 7f7e9f7ff700 1 Processor -- start 2023-07-06T21:41:11.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.570+0000 7f7e9f7ff700 1 -- start start 2023-07-06T21:41:11.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9f7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e980b81c0 0x7f7e980cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e980cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9f7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7e980ceda0 0x7f7e9815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9f7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980d1830 con 0x7f7e980b81c0 2023-07-06T21:41:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9f7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e980cf330 con 0x7f7e980ceda0 2023-07-06T21:41:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980d1270 con 0x7f7e980b1a80 2023-07-06T21:41:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9d59b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e980b81c0 0x7f7e980cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9d59b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7e980b81c0 0x7f7e980cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38854/0 (socket says 172.21.15.84:38854) 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9d59b700 1 -- 172.21.15.84:0/1273891988 learned_addr learned my addr 172.21.15.84:0/1273891988 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9cd9a700 1 --2- 172.21.15.84:0/1273891988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e980cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.571+0000 7f7e9dd9c700 1 --2- 172.21.15.84:0/1273891988 >> v2:172.21.15.107:3300/0 conn(0x7f7e980ceda0 0x7f7e9815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e9d59b700 1 -- 172.21.15.84:0/1273891988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 msgr2=0x7f7e980cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e9d59b700 1 --2- 172.21.15.84:0/1273891988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e980cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e9d59b700 1 -- 172.21.15.84:0/1273891988 >> v2:172.21.15.107:3300/0 conn(0x7f7e980ceda0 msgr2=0x7f7e9815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:11.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e9d59b700 1 --2- 172.21.15.84:0/1273891988 >> v2:172.21.15.107:3300/0 conn(0x7f7e980ceda0 0x7f7e9815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e9d59b700 1 -- 172.21.15.84:0/1273891988 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e980d1550 con 0x7f7e980b81c0 2023-07-06T21:41:11.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e9d59b700 1 --2- 172.21.15.84:0/1273891988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980b81c0 0x7f7e980cf600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7e980b8d30 tx=0x7f7e94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728686a2f1ca11d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:11.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e8ffff700 1 -- 172.21.15.84:0/1273891988 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400b870 con 0x7f7e980b81c0 2023-07-06T21:41:11.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e8ffff700 1 -- 172.21.15.84:0/1273891988 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9400b9d0 con 0x7f7e980b81c0 2023-07-06T21:41:11.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.572+0000 7f7e8ffff700 1 -- 172.21.15.84:0/1273891988 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9400bd60 con 0x7f7e980b81c0 2023-07-06T21:41:11.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1273891988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980b81c0 msgr2=0x7f7e980cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1273891988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980b81c0 0x7f7e980cf600 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7e980b8d30 tx=0x7f7e94009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1273891988 shutdown_connections 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1273891988 >> v2:172.21.15.107:3300/0 conn(0x7f7e980ceda0 0x7f7e9815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1273891988 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980b81c0 0x7f7e980cf600 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1273891988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e980cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1273891988 >> 172.21.15.84:0/1273891988 conn(0x7f7e9806b2b0 msgr2=0x7f7e9806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:11.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1273891988 shutdown_connections 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.573+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1273891988 wait complete. 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.575+0000 7f7e9f7ff700 1 Processor -- start 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.575+0000 7f7e9f7ff700 1 -- start start 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e98154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9f7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e980b81c0 0x7f7e98141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9f7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 0x7f7e980c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9f7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e981421d0 con 0x7f7e980ceda0 2023-07-06T21:41:11.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9f7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980cad10 con 0x7f7e980b81c0 2023-07-06T21:41:11.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e980c9930 con 0x7f7e980b1a80 2023-07-06T21:41:11.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 0x7f7e980c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 0x7f7e980c6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38868/0 (socket says 172.21.15.84:38868) 2023-07-06T21:41:11.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 -- 172.21.15.84:0/1028398927 learned_addr learned my addr 172.21.15.84:0/1028398927 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:11.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9d59b700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e98154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9cd9a700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e980b81c0 0x7f7e98141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 -- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 msgr2=0x7f7e98154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e98154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 -- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e980b81c0 msgr2=0x7f7e98141c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e980b81c0 0x7f7e98141c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9dd9c700 1 -- 172.21.15.84:0/1028398927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e94009550 con 0x7f7e980ceda0 2023-07-06T21:41:11.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.576+0000 7f7e9cd9a700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e980b81c0 0x7f7e98141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:11.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.577+0000 7f7e9dd9c700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 0x7f7e980c6720 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7e90007e60 tx=0x7f7e9000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.577+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000ede0 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.577+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90014420 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.577+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e900147d0 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.577+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e980cafb0 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.577+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e980cb570 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.578+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e980cb760 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.579+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7e90014930 con 0x7f7e980ceda0 2023-07-06T21:41:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.580+0000 7f7e8dffb700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7e7c06ace0 0x7f7e7c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.580+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7e90089d10 con 0x7f7e980ceda0 2023-07-06T21:41:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.580+0000 7f7e9d59b700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7e7c06ace0 0x7f7e7c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.581+0000 7f7e9d59b700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7e7c06ace0 0x7f7e7c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e980b4490 tx=0x7f7e94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:11.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.584+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7e900586c0 con 0x7f7e980ceda0 2023-07-06T21:41:11.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.780+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f7e9804e1d0 con 0x7f7e980ceda0 2023-07-06T21:41:11.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.780+0000 7f7e8dffb700 1 -- 172.21.15.84:0/1028398927 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7f7e90058250 con 0x7f7e980ceda0 2023-07-06T21:41:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.782+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7e7c06ace0 msgr2=0x7f7e7c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.782+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7e7c06ace0 0x7f7e7c06d190 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e980b4490 tx=0x7f7e94008030 comp rx=0 tx=0).stop 2023-07-06T21:41:11.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.782+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 msgr2=0x7f7e980c6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.782+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 0x7f7e980c6720 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7e90007e60 tx=0x7f7e9000e3c0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 shutdown_connections 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7e7c06ace0 0x7f7e7c06d190 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7e980ceda0 0x7f7e980c6720 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e980b1a80 0x7f7e98154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 --2- 172.21.15.84:0/1028398927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e980b81c0 0x7f7e98141c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:11.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 >> 172.21.15.84:0/1028398927 conn(0x7f7e9806b2b0 msgr2=0x7f7e980d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 shutdown_connections 2023-07-06T21:41:11.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.783+0000 7f7e9f7ff700 1 -- 172.21.15.84:0/1028398927 wait complete. 2023-07-06T21:41:11.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:41:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.997+0000 7fa35e3ff700 1 Processor -- start 2023-07-06T21:41:11.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.997+0000 7fa35e3ff700 1 -- start start 2023-07-06T21:41:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa35e3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3580b1fd0 0x7fa3580cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa35e3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa3580b59c0 0x7fa35815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa35e3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3580d1460 con 0x7fa3580d2690 2023-07-06T21:41:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa35e3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580d2530 con 0x7fa3580b1fd0 2023-07-06T21:41:12.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa35e3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3580d1740 con 0x7fa3580b59c0 2023-07-06T21:41:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa357fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3580b1fd0 0x7fa3580cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa357fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fa3580b1fd0 0x7fa3580cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38884/0 (socket says 172.21.15.84:38884) 2023-07-06T21:41:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.998+0000 7fa357fff700 1 -- 172.21.15.84:0/215482106 learned_addr learned my addr 172.21.15.84:0/215482106 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa35c99c700 1 --2- 172.21.15.84:0/215482106 >> v2:172.21.15.107:3300/0 conn(0x7fa3580b59c0 0x7fa35815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa357fff700 1 -- 172.21.15.84:0/215482106 >> v2:172.21.15.107:3300/0 conn(0x7fa3580b59c0 msgr2=0x7fa35815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa357fff700 1 --2- 172.21.15.84:0/215482106 >> v2:172.21.15.107:3300/0 conn(0x7fa3580b59c0 0x7fa35815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa357fff700 1 -- 172.21.15.84:0/215482106 >> v1:172.21.15.107:6789/0 conn(0x7fa3580d2690 legacy=0x7fa3580cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa357fff700 1 -- 172.21.15.84:0/215482106 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580cd260 con 0x7fa3580b1fd0 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa35c99c700 1 --2- 172.21.15.84:0/215482106 >> v2:172.21.15.107:3300/0 conn(0x7fa3580b59c0 0x7fa35815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:12.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa357fff700 1 --2- 172.21.15.84:0/215482106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580b1fd0 0x7fa3580cf160 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa3580b8cb0 tx=0x7fa348009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcaca0162009ced server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa356ffd700 1 -- 172.21.15.84:0/215482106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348004230 con 0x7fa3580b1fd0 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa356ffd700 1 -- 172.21.15.84:0/215482106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348004390 con 0x7fa3580b1fd0 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:11.999+0000 7fa356ffd700 1 -- 172.21.15.84:0/215482106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3480036a0 con 0x7fa3580b1fd0 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 -- 172.21.15.84:0/215482106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580b1fd0 msgr2=0x7fa3580cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/215482106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580b1fd0 0x7fa3580cf160 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa3580b8cb0 tx=0x7fa348009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 -- 172.21.15.84:0/215482106 shutdown_connections 2023-07-06T21:41:12.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/215482106 >> v2:172.21.15.107:3300/0 conn(0x7fa3580b59c0 0x7fa35815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/215482106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580b1fd0 0x7fa3580cf160 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 -- 172.21.15.84:0/215482106 >> 172.21.15.84:0/215482106 conn(0x7fa35806b340 msgr2=0x7fa35806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 -- 172.21.15.84:0/215482106 shutdown_connections 2023-07-06T21:41:12.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.000+0000 7fa35e3ff700 1 -- 172.21.15.84:0/215482106 wait complete. 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.002+0000 7fa35e3ff700 1 Processor -- start 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.002+0000 7fa35e3ff700 1 -- start start 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3580b1fd0 0x7fa35814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35e3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3580b59c0 0x7fa35815ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35e3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 0x7fa3580b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35e3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35815b450 con 0x7fa3580d2690 2023-07-06T21:41:12.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35e3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3581e5b10 con 0x7fa3580b59c0 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35814e3d0 con 0x7fa3580b1fd0 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 0x7fa3580b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 0x7fa3580b6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38886/0 (socket says 172.21.15.84:38886) 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 -- 172.21.15.84:0/3431168570 learned_addr learned my addr 172.21.15.84:0/3431168570 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa357fff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3580b1fd0 0x7fa35814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa3577fe700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3580b59c0 0x7fa35815ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:12.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 -- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3580b1fd0 msgr2=0x7fa35814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3580b1fd0 0x7fa35814de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 -- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3580b59c0 msgr2=0x7fa35815ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3580b59c0 0x7fa35815ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.003+0000 7fa35c99c700 1 -- 172.21.15.84:0/3431168570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348009550 con 0x7fa3580d2690 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.004+0000 7fa35c99c700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 0x7fa3580b6120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa35000cd50 tx=0x7fa35000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.004+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000cea0 con 0x7fa3580d2690 2023-07-06T21:41:12.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.004+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35000f480 con 0x7fa3580d2690 2023-07-06T21:41:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.004+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3581e4f00 con 0x7fa3580d2690 2023-07-06T21:41:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.004+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35000f830 con 0x7fa3580d2690 2023-07-06T21:41:12.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.004+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3581e54c0 con 0x7fa3580d2690 2023-07-06T21:41:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.005+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3581e5cc0 con 0x7fa3580d2690 2023-07-06T21:41:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.006+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa35000b040 con 0x7fa3580d2690 2023-07-06T21:41:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.007+0000 7fa354ff9700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa34406ace0 0x7fa34406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.007+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa350089d30 con 0x7fa3580d2690 2023-07-06T21:41:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.008+0000 7fa357fff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa34406ace0 0x7fa34406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:12.011 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.008+0000 7fa357fff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa34406ace0 0x7fa34406d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa348009130 tx=0x7fa348008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:12.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.012+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa3581e5cc0 con 0x7fa3580d2690 2023-07-06T21:41:12.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.224+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa3581e5c80 con 0x7fa3580d2690 2023-07-06T21:41:12.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.225+0000 7fa354ff9700 1 -- 172.21.15.84:0/3431168570 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa350054bb0 con 0x7fa3580d2690 2023-07-06T21:41:12.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa34406ace0 msgr2=0x7fa34406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:12.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa34406ace0 0x7fa34406d190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa348009130 tx=0x7fa348008030 comp rx=0 tx=0).stop 2023-07-06T21:41:12.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 msgr2=0x7fa3580b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 0x7fa3580b6120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa35000cd50 tx=0x7fa35000a560 comp rx=0 tx=0).stop 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 shutdown_connections 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa34406ace0 0x7fa34406d190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa3580d2690 0x7fa3580b6120 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3580b1fd0 0x7fa35814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 --2- 172.21.15.84:0/3431168570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3580b59c0 0x7fa35815ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 >> 172.21.15.84:0/3431168570 conn(0x7fa35806b340 msgr2=0x7fa3580b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:12.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 shutdown_connections 2023-07-06T21:41:12.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:12.227+0000 7fa35e3ff700 1 -- 172.21.15.84:0/3431168570 wait complete. 2023-07-06T21:41:12.231 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:41:12.247 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:41:12.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:12.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:12.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[147170]: fuse finished with error 0 and tester_r 0 2023-07-06T21:41:12.314 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:41:18.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:41:18.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:41:18.417 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:41:18.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:18.417 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:41:18.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:41:18.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.661+0000 7f6038fff700 1 Processor -- start 2023-07-06T21:41:18.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.661+0000 7f6038fff700 1 -- start start 2023-07-06T21:41:18.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6038fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f60340b8530 0x7f60340cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:18.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6038fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60340d3580 0x7f60340d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:18.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6038fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60340d0990 con 0x7f60340cf3b0 2023-07-06T21:41:18.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6038fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340d3af0 con 0x7f60340d3580 2023-07-06T21:41:18.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6038fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340cd9d0 con 0x7f60340b8530 2023-07-06T21:41:18.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6031d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60340d3580 0x7f60340d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6031d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f60340d3580 0x7f60340d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53332/0 (socket says 172.21.15.84:53332) 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f6031d9b700 1 -- 172.21.15.84:0/1456618912 learned_addr learned my addr 172.21.15.84:0/1456618912 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.662+0000 7f603259c700 1 --2- 172.21.15.84:0/1456618912 >> v2:172.21.15.107:3300/0 conn(0x7f60340b8530 0x7f60340cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f6031d9b700 1 -- 172.21.15.84:0/1456618912 >> v2:172.21.15.107:3300/0 conn(0x7f60340b8530 msgr2=0x7f60340cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f6031d9b700 1 --2- 172.21.15.84:0/1456618912 >> v2:172.21.15.107:3300/0 conn(0x7f60340b8530 0x7f60340cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f6031d9b700 1 -- 172.21.15.84:0/1456618912 >> v1:172.21.15.107:6789/0 conn(0x7f60340cf3b0 legacy=0x7f603415feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f6031d9b700 1 -- 172.21.15.84:0/1456618912 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340cd410 con 0x7f60340d3580 2023-07-06T21:41:18.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f6031d9b700 1 --2- 172.21.15.84:0/1456618912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340d3580 0x7f60340d0e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f603406c680 tx=0x7f6028009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8d6b4c14cbf730 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:18.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f603159a700 1 -- 172.21.15.84:0/1456618912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800b870 con 0x7f60340d3580 2023-07-06T21:41:18.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f603159a700 1 -- 172.21.15.84:0/1456618912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800b9d0 con 0x7f60340d3580 2023-07-06T21:41:18.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.663+0000 7f603159a700 1 -- 172.21.15.84:0/1456618912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800bd60 con 0x7f60340d3580 2023-07-06T21:41:18.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 -- 172.21.15.84:0/1456618912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340d3580 msgr2=0x7f60340d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 --2- 172.21.15.84:0/1456618912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340d3580 0x7f60340d0e70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f603406c680 tx=0x7f6028009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 -- 172.21.15.84:0/1456618912 shutdown_connections 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 --2- 172.21.15.84:0/1456618912 >> v2:172.21.15.107:3300/0 conn(0x7f60340b8530 0x7f60340cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 --2- 172.21.15.84:0/1456618912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340d3580 0x7f60340d0e70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 -- 172.21.15.84:0/1456618912 >> 172.21.15.84:0/1456618912 conn(0x7f60340715c0 msgr2=0x7f6034071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 -- 172.21.15.84:0/1456618912 shutdown_connections 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.664+0000 7f6038fff700 1 -- 172.21.15.84:0/1456618912 wait complete. 2023-07-06T21:41:18.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.666+0000 7f6038fff700 1 Processor -- start 2023-07-06T21:41:18.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.666+0000 7f6038fff700 1 -- start start 2023-07-06T21:41:18.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6038fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60340b8530 0x7f60340cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:18.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6038fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 0x7f60340ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:18.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6038fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60340d3580 0x7f6034150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:18.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6038fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340cacf0 con 0x7f60340cf3b0 2023-07-06T21:41:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6038fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60340caf40 con 0x7f60340b8530 2023-07-06T21:41:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6038fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340cb220 con 0x7f60340d3580 2023-07-06T21:41:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 0x7f60340ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 0x7f60340ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53338/0 (socket says 172.21.15.84:53338) 2023-07-06T21:41:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 -- 172.21.15.84:0/1730717436 learned_addr learned my addr 172.21.15.84:0/1730717436 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:18.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 -- 172.21.15.84:0/1730717436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60340d3580 msgr2=0x7f6034150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60340d3580 0x7f6034150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 -- 172.21.15.84:0/1730717436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60340b8530 msgr2=0x7f60340cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60340b8530 0x7f60340cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 -- 172.21.15.84:0/1730717436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6028009550 con 0x7f60340cf3b0 2023-07-06T21:41:18.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.667+0000 7f6031d9b700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 0x7f60340ccbb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.668+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028004520 con 0x7f60340cf3b0 2023-07-06T21:41:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.668+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602801e070 con 0x7f60340cf3b0 2023-07-06T21:41:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.668+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60340cb500 con 0x7f60340cf3b0 2023-07-06T21:41:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.668+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280038a0 con 0x7f60340cf3b0 2023-07-06T21:41:18.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.668+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60340cbac0 con 0x7f60340cf3b0 2023-07-06T21:41:18.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.669+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60340b4c00 con 0x7f60340cf3b0 2023-07-06T21:41:18.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.670+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6028003a00 con 0x7f60340cf3b0 2023-07-06T21:41:18.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.671+0000 7f6022ffd700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f601406ad10 0x7f601406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:18.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.671+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6028020070 con 0x7f60340cf3b0 2023-07-06T21:41:18.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.671+0000 7f603259c700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f601406ad10 0x7f601406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.671+0000 7f603259c700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f601406ad10 0x7f601406d1c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6024006eb0 tx=0x7f6024008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:18.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.675+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6028055b10 con 0x7f60340cf3b0 2023-07-06T21:41:18.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.916+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60340cbce0 con 0x7f60340cf3b0 2023-07-06T21:41:18.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.917+0000 7f6022ffd700 1 -- 172.21.15.84:0/1730717436 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+82 (secure 0 0 0) 0x7f6028015070 con 0x7f60340cf3b0 2023-07-06T21:41:18.919 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-07-06T21:41:18.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f601406ad10 msgr2=0x7f601406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f601406ad10 0x7f601406d1c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6024006eb0 tx=0x7f6024008040 comp rx=0 tx=0).stop 2023-07-06T21:41:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 msgr2=0x7f60340ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 0x7f60340ccbb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028004750 comp rx=0 tx=0).stop 2023-07-06T21:41:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 shutdown_connections 2023-07-06T21:41:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f601406ad10 0x7f601406d1c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60340d3580 0x7f6034150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60340b8530 0x7f60340cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 --2- 172.21.15.84:0/1730717436 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f60340cf3b0 0x7f60340ccbb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 >> 172.21.15.84:0/1730717436 conn(0x7f60340715c0 msgr2=0x7f603415ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 shutdown_connections 2023-07-06T21:41:18.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:18.919+0000 7f6038fff700 1 -- 172.21.15.84:0/1730717436 wait complete. 2023-07-06T21:41:18.939 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:41:18.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:41:18.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:41:18.941 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:41:18.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:18.941 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:41:18.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:41:18.942 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:41:18.942 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:18.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:18.943 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:41:18.988 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:41:18.988 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:41:18.989 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:41:18.989 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.990 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:41:18.990 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.990 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:18.991 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:18.991 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:41:18.991 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:18.991 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:18.992 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:41:18.992 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:18.992 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:18.993 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:41:18.993 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:18.993 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:41:18.993 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42123sec preferred_lft 42123sec 2023-07-06T21:41:18.993 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:41:18.993 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.994 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4492:bbff:fe16:eebb/64 scope link 2023-07-06T21:41:18.995 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:18.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:18.996 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:41:19.093 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:41:19.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:19.096 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:41:19.141 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:41:19.142 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:41:19.142 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:41:19.142 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:41:19.142 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:41:19.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:19.144 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:41:19.144 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:41:19.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:41:19.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:41:19.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:41:19.145 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:41:19.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:19.351 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:41:19.393 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:41:19.393 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:41:19.394 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:19.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:19.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:41:19.502 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:41:19.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:41:19.504 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:19.546 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:19.546 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:41:19.547 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:19.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:41:19.736 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:41:19.736 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:41:19.802 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:41:19.805 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:41:19.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:19.806 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:41:19.848 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:41:19.849 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:41:19.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:41:19.962 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:41:19.962 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:41:20.029 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:41:20.032 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:41:20.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:20.033 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:41:20.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:41:20.058+0000 7f0e4d2825c0 -1 init, newargv = 0x559293bf5b70 newargc=15 2023-07-06T21:41:20.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[148692]: starting ceph client 2023-07-06T21:41:20.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[148692]: starting fuse 2023-07-06T21:41:20.080 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:41:20.080 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:41:20.081 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:41:20.082 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:41:20.083 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:41:20.083 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:41:20.084 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:41:20.085 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:20.086 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:41:20.086 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:41:20.087 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:41:20.087 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:41:20.088 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:20.089 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:41:20.089 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:41:20.090 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:41:20.090 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:41:20.091 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:20.091 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:41:20.091 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:41:20.092 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:41:20.092 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:41:20.092 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:41:20.093 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:41:20.093 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:41:20.093 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:41:20.094 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:20.094 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:41:20.094 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:41:20.261 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.148692.asok 2023-07-06T21:41:20.439 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.148692.asok 2023-07-06T21:41:20.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:20.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.148692.asok status 2023-07-06T21:41:20.664 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:41:20.664 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:41:20.664 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:41:20.664 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "148692", 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:20.665 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5652, 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 5652 2023-07-06T21:41:20.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3641338691 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:41:20.667 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:41:20.668 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:41:20.668 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3641338691 2023-07-06T21:41:20.668 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:20.668 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.5652 192.168.0.1:0/3641338691", 2023-07-06T21:41:20.668 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/3641338691", 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 16, 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 40, 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:41:20.669 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:41:20.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:20.678 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:41:20.679 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:20.723 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:41:20.723 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:20.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:20.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:20.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:20.830 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-07-06T21:41:20.879 INFO:teuthology.orchestra.run.smithi084.stderr:setfattr: layoutfile: Permission denied 2023-07-06T21:41:20.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:41:20.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:20.881 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-07-06T21:41:20.925 INFO:teuthology.orchestra.run.smithi084.stderr:setfattr: layoutdir: Permission denied 2023-07-06T21:41:20.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:41:20.926 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:20.927 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:21.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:21.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.243+0000 7f2bda9ff700 1 Processor -- start 2023-07-06T21:41:21.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.243+0000 7f2bda9ff700 1 -- start start 2023-07-06T21:41:21.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.244+0000 7f2bda9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2bd40b3010 0x7f2bd406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.244+0000 7f2bda9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd406b8a0 con 0x7f2bd40b2470 2023-07-06T21:41:21.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.244+0000 7f2bda9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd406bb80 con 0x7f2bd415c6a0 2023-07-06T21:41:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.244+0000 7f2bda9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd415ca70 con 0x7f2bd40b3010 2023-07-06T21:41:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.244+0000 7f2bd37fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2bd40b3010 0x7f2bd406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.244+0000 7f2bd37fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2bd40b3010 0x7f2bd406afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53346/0 (socket says 172.21.15.84:53346) 2023-07-06T21:41:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd37fe700 1 -- 172.21.15.84:0/4291001808 learned_addr learned my addr 172.21.15.84:0/4291001808 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd37fe700 1 -- 172.21.15.84:0/4291001808 >> v1:172.21.15.107:6789/0 conn(0x7f2bd415c6a0 legacy=0x7f2bd406c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd37fe700 1 -- 172.21.15.84:0/4291001808 >> v1:172.21.15.84:6789/0 conn(0x7f2bd40b2470 legacy=0x7f2bd40cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd37fe700 1 -- 172.21.15.84:0/4291001808 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd406be60 con 0x7f2bd40b3010 2023-07-06T21:41:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd37fe700 1 --2- 172.21.15.84:0/4291001808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd40b3010 0x7f2bd406afa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098f0 tx=0x7f2bc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4e767e93566e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd2ffd700 1 -- 172.21.15.84:0/4291001808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800b8a0 con 0x7f2bd40b3010 2023-07-06T21:41:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd2ffd700 1 -- 172.21.15.84:0/4291001808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800ba00 con 0x7f2bd40b3010 2023-07-06T21:41:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.245+0000 7f2bd2ffd700 1 -- 172.21.15.84:0/4291001808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800bd90 con 0x7f2bd40b3010 2023-07-06T21:41:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 -- 172.21.15.84:0/4291001808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd40b3010 msgr2=0x7f2bd406afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/4291001808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd40b3010 0x7f2bd406afa0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098f0 tx=0x7f2bc8009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 -- 172.21.15.84:0/4291001808 shutdown_connections 2023-07-06T21:41:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/4291001808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd40b3010 0x7f2bd406afa0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 -- 172.21.15.84:0/4291001808 >> 172.21.15.84:0/4291001808 conn(0x7f2bd406e640 msgr2=0x7f2bd406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 -- 172.21.15.84:0/4291001808 shutdown_connections 2023-07-06T21:41:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.246+0000 7f2bda9ff700 1 -- 172.21.15.84:0/4291001808 wait complete. 2023-07-06T21:41:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.248+0000 7f2bda9ff700 1 Processor -- start 2023-07-06T21:41:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.248+0000 7f2bda9ff700 1 -- start start 2023-07-06T21:41:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bda9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bd40b2470 0x7f2bd4151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bda9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd40b3010 0x7f2bd40c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bda9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 0x7f2bd40b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bda9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c5fc0 con 0x7f2bd415c6a0 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bda9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c62a0 con 0x7f2bd40b3010 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bda9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40c6580 con 0x7f2bd40b2470 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bd8f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 0x7f2bd40b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bd8f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 0x7f2bd40b3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53348/0 (socket says 172.21.15.84:53348) 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bd8f9c700 1 -- 172.21.15.84:0/1349623751 learned_addr learned my addr 172.21.15.84:0/1349623751 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bd3fff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bd40b2470 0x7f2bd4151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.249+0000 7f2bd37fe700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd40b3010 0x7f2bd40c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd8f9c700 1 -- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bd40b2470 msgr2=0x7f2bd4151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd8f9c700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bd40b2470 0x7f2bd4151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd8f9c700 1 -- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd40b3010 msgr2=0x7f2bd40c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd8f9c700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd40b3010 0x7f2bd40c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd8f9c700 1 -- 172.21.15.84:0/1349623751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc8009550 con 0x7f2bd415c6a0 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd8f9c700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 0x7f2bd40b3820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00cc50 tx=0x7f2bcc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc003c00 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bcc003d60 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd40c6860 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc00ab60 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.250+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd4152390 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.251+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd4152520 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.252+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2bcc00acc0 con 0x7f2bd415c6a0 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.253+0000 7f2bd0ff9700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2bc006ad10 0x7f2bc006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.253+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2bcc089770 con 0x7f2bd415c6a0 2023-07-06T21:41:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.253+0000 7f2bd3fff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2bc006ad10 0x7f2bc006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.254+0000 7f2bd3fff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2bc006ad10 0x7f2bc006d1c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4000f60 tx=0x7f2bc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.258+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2bcc0545f0 con 0x7f2bd415c6a0 2023-07-06T21:41:21.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.454+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bd40c6d80 con 0x7f2bd415c6a0 2023-07-06T21:41:21.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.456+0000 7f2bd0ff9700 1 -- 172.21.15.84:0/1349623751 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7f2bd40c6d80 con 0x7f2bd415c6a0 2023-07-06T21:41:21.458 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:21.458 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2bc006ad10 msgr2=0x7f2bc006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2bc006ad10 0x7f2bc006d1c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4000f60 tx=0x7f2bc4009040 comp rx=0 tx=0).stop 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 msgr2=0x7f2bd40b3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 0x7f2bd40b3820 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00cc50 tx=0x7f2bcc00cf60 comp rx=0 tx=0).stop 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 shutdown_connections 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2bc006ad10 0x7f2bc006d1c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd415c6a0 0x7f2bd40b3820 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bd40b2470 0x7f2bd4151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 --2- 172.21.15.84:0/1349623751 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bd40b3010 0x7f2bd40c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 >> 172.21.15.84:0/1349623751 conn(0x7f2bd406e640 msgr2=0x7f2bd40cd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:21.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.458+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 shutdown_connections 2023-07-06T21:41:21.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.459+0000 7f2bda9ff700 1 -- 172.21.15.84:0/1349623751 wait complete. 2023-07-06T21:41:21.479 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:21.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.673+0000 7f1d629ff700 1 Processor -- start 2023-07-06T21:41:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.674+0000 7f1d629ff700 1 -- start start 2023-07-06T21:41:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d629ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1d5c0cfd10 0x7f1d5c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d629ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d5c0b1a80 0x7f1d5c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d629ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5c0d1830 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d629ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c0cf330 con 0x7f1d5c0b1a80 2023-07-06T21:41:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d629ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5c0d1270 con 0x7f1d5c0cfd10 2023-07-06T21:41:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d60f9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d5c0b1a80 0x7f1d5c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d60f9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1d5c0b1a80 0x7f1d5c0d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53358/0 (socket says 172.21.15.84:53358) 2023-07-06T21:41:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d60f9c700 1 -- 172.21.15.84:0/1572182768 learned_addr learned my addr 172.21.15.84:0/1572182768 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.675+0000 7f1d5b7fe700 1 --2- 172.21.15.84:0/1572182768 >> v2:172.21.15.107:3300/0 conn(0x7f1d5c0cfd10 0x7f1d5c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d60f9c700 1 -- 172.21.15.84:0/1572182768 >> v2:172.21.15.107:3300/0 conn(0x7f1d5c0cfd10 msgr2=0x7f1d5c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d60f9c700 1 --2- 172.21.15.84:0/1572182768 >> v2:172.21.15.107:3300/0 conn(0x7f1d5c0cfd10 0x7f1d5c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d60f9c700 1 -- 172.21.15.84:0/1572182768 >> v1:172.21.15.107:6789/0 conn(0x7f1d5c0b81c0 legacy=0x7f1d5c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d60f9c700 1 -- 172.21.15.84:0/1572182768 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d5c0d1550 con 0x7f1d5c0b1a80 2023-07-06T21:41:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d60f9c700 1 --2- 172.21.15.84:0/1572182768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b1a80 0x7f1d5c0d28c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0d0770 tx=0x7f1d5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0bb2dfc3eb1397 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d5affd700 1 -- 172.21.15.84:0/1572182768 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400e070 con 0x7f1d5c0b1a80 2023-07-06T21:41:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d5affd700 1 -- 172.21.15.84:0/1572182768 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5400b9c0 con 0x7f1d5c0b1a80 2023-07-06T21:41:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.676+0000 7f1d5affd700 1 -- 172.21.15.84:0/1572182768 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5400bd70 con 0x7f1d5c0b1a80 2023-07-06T21:41:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 -- 172.21.15.84:0/1572182768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b1a80 msgr2=0x7f1d5c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 --2- 172.21.15.84:0/1572182768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b1a80 0x7f1d5c0d28c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0d0770 tx=0x7f1d5400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 -- 172.21.15.84:0/1572182768 shutdown_connections 2023-07-06T21:41:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 --2- 172.21.15.84:0/1572182768 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b1a80 0x7f1d5c0d28c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 --2- 172.21.15.84:0/1572182768 >> v2:172.21.15.107:3300/0 conn(0x7f1d5c0cfd10 0x7f1d5c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 -- 172.21.15.84:0/1572182768 >> 172.21.15.84:0/1572182768 conn(0x7f1d5c06b2b0 msgr2=0x7f1d5c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 -- 172.21.15.84:0/1572182768 shutdown_connections 2023-07-06T21:41:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.677+0000 7f1d629ff700 1 -- 172.21.15.84:0/1572182768 wait complete. 2023-07-06T21:41:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.679+0000 7f1d629ff700 1 Processor -- start 2023-07-06T21:41:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.679+0000 7f1d629ff700 1 -- start start 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.680+0000 7f1d629ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d5c0b1a80 0x7f1d5c14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d629ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 0x7f1d5c1e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d629ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c0cfd10 0x7f1d5c1f49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d629ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1f3910 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d629ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1f3bf0 con 0x7f1d5c0cfd10 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d629ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5c1f3ed0 con 0x7f1d5c0b1a80 2023-07-06T21:41:21.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 0x7f1d5c1e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 0x7f1d5c1e2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53368/0 (socket says 172.21.15.84:53368) 2023-07-06T21:41:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 -- 172.21.15.84:0/2450815690 learned_addr learned my addr 172.21.15.84:0/2450815690 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5bfff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d5c0b1a80 0x7f1d5c14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d60f9c700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c0cfd10 0x7f1d5c1f49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 -- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d5c0b1a80 msgr2=0x7f1d5c14def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d5c0b1a80 0x7f1d5c14def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 -- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c0cfd10 msgr2=0x7f1d5c1f49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c0cfd10 0x7f1d5c1f49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 -- 172.21.15.84:0/2450815690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d54009b30 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d60f9c700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c0cfd10 0x7f1d5c1f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5bfff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d5c0b1a80 0x7f1d5c14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.681+0000 7f1d5b7fe700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 0x7f1d5c1e2a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000d7d0 tx=0x7f1d5000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:21.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.682+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d500125f0 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.682+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d50012790 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.682+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5c1f41b0 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.682+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50012b40 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.682+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5c14e430 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.683+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d5c1f46d0 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.684+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1d50012ca0 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.685+0000 7f1d58ff9700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1d4806acb0 0x7f1d4806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.685+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1d50089b40 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.685+0000 7f1d5bfff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1d4806acb0 0x7f1d4806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:21.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.685+0000 7f1d5bfff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1d4806acb0 0x7f1d4806d160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c000f60 tx=0x7f1d4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:21.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.689+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1d50054930 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.887+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d5c04e1d0 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.888+0000 7f1d58ff9700 1 -- 172.21.15.84:0/2450815690 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d50057f50 con 0x7f1d5c0b81c0 2023-07-06T21:41:21.890 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1d4806acb0 msgr2=0x7f1d4806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1d4806acb0 0x7f1d4806d160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c000f60 tx=0x7f1d4c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 msgr2=0x7f1d5c1e2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:21.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 0x7f1d5c1e2a50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000d7d0 tx=0x7f1d5000db90 comp rx=0 tx=0).stop 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 shutdown_connections 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1d4806acb0 0x7f1d4806d160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d5c0cfd10 0x7f1d5c1f49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d5c0b1a80 0x7f1d5c14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 --2- 172.21.15.84:0/2450815690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1d5c0b81c0 0x7f1d5c1e2a50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 >> 172.21.15.84:0/2450815690 conn(0x7f1d5c06b2b0 msgr2=0x7f1d5c0c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 shutdown_connections 2023-07-06T21:41:21.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:21.890+0000 7f1d629ff700 1 -- 172.21.15.84:0/2450815690 wait complete. 2023-07-06T21:41:21.910 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:22.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.105+0000 7fa9debff700 1 Processor -- start 2023-07-06T21:41:22.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.105+0000 7fa9debff700 1 -- start start 2023-07-06T21:41:22.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.106+0000 7fa9debff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 0x7fa9d80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.106+0000 7fa9debff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d80d1830 con 0x7fa9d80b1a80 2023-07-06T21:41:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.106+0000 7fa9debff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d8160d40 con 0x7fa9d80ceda0 2023-07-06T21:41:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.106+0000 7fa9debff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80cf330 con 0x7fa9d80b81c0 2023-07-06T21:41:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.106+0000 7fa9d7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa9d80b1a80 0x7fa9d815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58016/0 (socket says 172.21.15.84:58016) 2023-07-06T21:41:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.106+0000 7fa9d7fff700 1 -- 172.21.15.84:0/893215259 learned_addr learned my addr 172.21.15.84:0/893215259 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 0x7fa9d80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084596568 0 0) 0x7fa9d80d1830 con 0x7fa9d80b1a80 2023-07-06T21:41:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9c80035b0 con 0x7fa9d80b1a80 2023-07-06T21:41:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510417695 0 0) 0x7fa9d8160d40 con 0x7fa9d80ceda0 2023-07-06T21:41:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d80d1830 con 0x7fa9d80ceda0 2023-07-06T21:41:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626487753 0 0) 0x7fa9c80035b0 con 0x7fa9d80b1a80 2023-07-06T21:41:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 msgr2=0x7fa9d80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 0x7fa9d80cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 >> v1:172.21.15.107:6789/0 conn(0x7fa9d80ceda0 legacy=0x7fa9d8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d80d1270 con 0x7fa9d80b1a80 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fa9c4001670 con 0x7fa9d80b1a80 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 >> v1:172.21.15.84:6789/0 conn(0x7fa9d80b1a80 legacy=0x7fa9d815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9c8003bc0 0x7fa9c8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 0x7fa9c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d80cf330 con 0x7fa9c8003bc0 2023-07-06T21:41:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c800bac0 con 0x7fa9c8004d90 2023-07-06T21:41:22.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.107+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c800bc80 con 0x7fa9d80b81c0 2023-07-06T21:41:22.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9c8003bc0 0x7fa9c8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9d7fff700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 0x7fa9c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dd19c700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 -- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 msgr2=0x7fa9c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:22.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d80b81c0 0x7fa9c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 -- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9c8004d90 msgr2=0x7fa9c8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:22.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9c8004d90 0x7fa9c8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 -- 172.21.15.84:0/893215259 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c80035b0 con 0x7fa9c8003bc0 2023-07-06T21:41:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9c8003bc0 0x7fa9c8003fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009b30 tx=0x7fa9c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c717add8aeae7d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00e6c0 con 0x7fa9c8003bc0 2023-07-06T21:41:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00e820 con 0x7fa9c8003bc0 2023-07-06T21:41:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.108+0000 7fa9d77fe700 1 -- 172.21.15.84:0/893215259 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00eb90 con 0x7fa9c8003bc0 2023-07-06T21:41:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9debff700 1 -- 172.21.15.84:0/893215259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9c8003bc0 msgr2=0x7fa9c8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9debff700 1 --2- 172.21.15.84:0/893215259 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9c8003bc0 0x7fa9c8003fb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009b30 tx=0x7fa9c8005880 comp rx=0 tx=0).stop 2023-07-06T21:41:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9dd19c700 1 -- 172.21.15.84:0/893215259 reap_dead start 2023-07-06T21:41:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9debff700 1 -- 172.21.15.84:0/893215259 shutdown_connections 2023-07-06T21:41:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9debff700 1 -- 172.21.15.84:0/893215259 >> 172.21.15.84:0/893215259 conn(0x7fa9d806b2b0 msgr2=0x7fa9d806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9debff700 1 -- 172.21.15.84:0/893215259 shutdown_connections 2023-07-06T21:41:22.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.109+0000 7fa9debff700 1 -- 172.21.15.84:0/893215259 wait complete. 2023-07-06T21:41:22.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.111+0000 7fa9debff700 1 Processor -- start 2023-07-06T21:41:22.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.111+0000 7fa9debff700 1 -- start start 2023-07-06T21:41:22.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9debff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9c8003bc0 0x7fa9d8154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9debff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 0x7fa9d80cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9debff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d80cbc90 0x7fa9d80715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9debff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81e68f0 con 0x7fa9d81e8280 2023-07-06T21:41:22.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9debff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81e5ce0 con 0x7fa9d80cbc90 2023-07-06T21:41:22.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9debff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81e5fc0 con 0x7fa9c8003bc0 2023-07-06T21:41:22.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 0x7fa9d80cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 0x7fa9d80cb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53390/0 (socket says 172.21.15.84:53390) 2023-07-06T21:41:22.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.112+0000 7fa9d7fff700 1 -- 172.21.15.84:0/642954655 learned_addr learned my addr 172.21.15.84:0/642954655 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:22.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9dd19c700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d80cbc90 0x7fa9d80715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d7fff700 1 -- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9c8003bc0 msgr2=0x7fa9d8154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9c8003bc0 0x7fa9d8154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d7fff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9c8003bc0 0x7fa9d8154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d7fff700 1 -- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d80cbc90 msgr2=0x7fa9d80715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d7fff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d80cbc90 0x7fa9d80715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d7fff700 1 -- 172.21.15.84:0/642954655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d81e62a0 con 0x7fa9d81e8280 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9dd19c700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d80cbc90 0x7fa9d80715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:22.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9c8003bc0 0x7fa9d8154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d7fff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 0x7fa9d80cb750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400e700 tx=0x7fa9c400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4012590 con 0x7fa9d81e8280 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c4012710 con 0x7fa9d81e8280 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d81e6580 con 0x7fa9d81e8280 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c4012ac0 con 0x7fa9d81e8280 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.113+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d8154650 con 0x7fa9d81e8280 2023-07-06T21:41:22.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.115+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d804e1d0 con 0x7fa9d81e8280 2023-07-06T21:41:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.115+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa9c4012c20 con 0x7fa9d81e8280 2023-07-06T21:41:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.116+0000 7fa9d57fa700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa9bc06ace0 0x7fa9bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.117+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fa9c4089400 con 0x7fa9d81e8280 2023-07-06T21:41:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.117+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa9bc06ace0 0x7fa9bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.117+0000 7fa9dc99b700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa9bc06ace0 0x7fa9bc06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc000c00 tx=0x7fa9cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.120+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa9c4054330 con 0x7fa9d81e8280 2023-07-06T21:41:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.319+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa9d80b4f30 con 0x7fa9d81e8280 2023-07-06T21:41:22.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.319+0000 7fa9d57fa700 1 -- 172.21.15.84:0/642954655 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa9c4012ed0 con 0x7fa9d81e8280 2023-07-06T21:41:22.322 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:22.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa9bc06ace0 msgr2=0x7fa9bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa9bc06ace0 0x7fa9bc06d190 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc000c00 tx=0x7fa9cc008030 comp rx=0 tx=0).stop 2023-07-06T21:41:22.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 msgr2=0x7fa9d80cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 0x7fa9d80cb750 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400e700 tx=0x7fa9c400eac0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 shutdown_connections 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa9bc06ace0 0x7fa9bc06d190 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d80cbc90 0x7fa9d80715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9c8003bc0 0x7fa9d8154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 --2- 172.21.15.84:0/642954655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9d81e8280 0x7fa9d80cb750 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 >> 172.21.15.84:0/642954655 conn(0x7fa9d806b2b0 msgr2=0x7fa9d80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 shutdown_connections 2023-07-06T21:41:22.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.322+0000 7fa9debff700 1 -- 172.21.15.84:0/642954655 wait complete. 2023-07-06T21:41:22.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.538+0000 7f9097dff700 1 Processor -- start 2023-07-06T21:41:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.538+0000 7f9097dff700 1 -- start start 2023-07-06T21:41:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f9097dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f90900cfd10 0x7f90900d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f9097dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90900b1a80 0x7f90900d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f9097dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90900d1830 con 0x7f90900b81c0 2023-07-06T21:41:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f9097dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900cf330 con 0x7f90900b1a80 2023-07-06T21:41:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f9097dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900d1270 con 0x7f90900cfd10 2023-07-06T21:41:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f909639c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90900b1a80 0x7f90900d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f909639c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f90900b1a80 0x7f90900d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53398/0 (socket says 172.21.15.84:53398) 2023-07-06T21:41:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f909639c700 1 -- 172.21.15.84:0/2041141414 learned_addr learned my addr 172.21.15.84:0/2041141414 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f909539a700 1 --2- 172.21.15.84:0/2041141414 >> v2:172.21.15.107:3300/0 conn(0x7f90900cfd10 0x7f90900d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f909639c700 1 -- 172.21.15.84:0/2041141414 >> v2:172.21.15.107:3300/0 conn(0x7f90900cfd10 msgr2=0x7f90900d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:22.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.540+0000 7f909639c700 1 --2- 172.21.15.84:0/2041141414 >> v2:172.21.15.107:3300/0 conn(0x7f90900cfd10 0x7f90900d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f909639c700 1 -- 172.21.15.84:0/2041141414 >> v1:172.21.15.107:6789/0 conn(0x7f90900b81c0 legacy=0x7f90900cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f909639c700 1 -- 172.21.15.84:0/2041141414 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900d1550 con 0x7f90900b1a80 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f909639c700 1 --2- 172.21.15.84:0/2041141414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900b1a80 0x7f90900d28c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f90900d0770 tx=0x7f908800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ff6b49adbeac58 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f9094b99700 1 -- 172.21.15.84:0/2041141414 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800e070 con 0x7f90900b1a80 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f9094b99700 1 -- 172.21.15.84:0/2041141414 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908800b9c0 con 0x7f90900b1a80 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f9094b99700 1 -- 172.21.15.84:0/2041141414 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800bd70 con 0x7f90900b1a80 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f9097dff700 1 -- 172.21.15.84:0/2041141414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900b1a80 msgr2=0x7f90900d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.541+0000 7f9097dff700 1 --2- 172.21.15.84:0/2041141414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900b1a80 0x7f90900d28c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f90900d0770 tx=0x7f908800b330 comp rx=0 tx=0).stop 2023-07-06T21:41:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.542+0000 7f9097dff700 1 -- 172.21.15.84:0/2041141414 shutdown_connections 2023-07-06T21:41:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.542+0000 7f9097dff700 1 --2- 172.21.15.84:0/2041141414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900b1a80 0x7f90900d28c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.542+0000 7f9097dff700 1 --2- 172.21.15.84:0/2041141414 >> v2:172.21.15.107:3300/0 conn(0x7f90900cfd10 0x7f90900d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.542+0000 7f9097dff700 1 -- 172.21.15.84:0/2041141414 >> 172.21.15.84:0/2041141414 conn(0x7f909006b2b0 msgr2=0x7f909006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.542+0000 7f9097dff700 1 -- 172.21.15.84:0/2041141414 shutdown_connections 2023-07-06T21:41:22.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.542+0000 7f9097dff700 1 -- 172.21.15.84:0/2041141414 wait complete. 2023-07-06T21:41:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.543+0000 7f9097dff700 1 Processor -- start 2023-07-06T21:41:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.543+0000 7f9097dff700 1 -- start start 2023-07-06T21:41:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.544+0000 7f9097dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90900b1a80 0x7f909015aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.544+0000 7f9097dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90900b81c0 0x7f90901e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.544+0000 7f9097dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 0x7f909015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.544+0000 7f9097dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901e1c80 con 0x7f90900cfd10 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.544+0000 7f9097dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90901e1f60 con 0x7f90900b1a80 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.544+0000 7f9097dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90901e2240 con 0x7f90900b81c0 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 0x7f909015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 0x7f909015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53410/0 (socket says 172.21.15.84:53410) 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 -- 172.21.15.84:0/2657433216 learned_addr learned my addr 172.21.15.84:0/2657433216 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909539a700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90900b81c0 0x7f90901e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f9095b9b700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90900b1a80 0x7f909015aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 -- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90900b81c0 msgr2=0x7f90901e2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90900b81c0 0x7f90901e2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 -- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90900b1a80 msgr2=0x7f909015aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90900b1a80 0x7f909015aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 -- 172.21.15.84:0/2657433216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9088009b30 con 0x7f90900cfd10 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f909639c700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 0x7f909015a750 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f909006bed0 tx=0x7f9088004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90880099b0 con 0x7f90900cfd10 2023-07-06T21:41:22.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90880092e0 con 0x7f90900cfd10 2023-07-06T21:41:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.545+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90901e2520 con 0x7f90900cfd10 2023-07-06T21:41:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.546+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908801c640 con 0x7f90900cfd10 2023-07-06T21:41:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.546+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909015ac90 con 0x7f90900cfd10 2023-07-06T21:41:22.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.547+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90901e29e0 con 0x7f90900cfd10 2023-07-06T21:41:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.548+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f908801c7a0 con 0x7f90900cfd10 2023-07-06T21:41:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.549+0000 7f90867fc700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f907406ace0 0x7f907406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.549+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f908808aff0 con 0x7f90900cfd10 2023-07-06T21:41:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.549+0000 7f9095b9b700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f907406ace0 0x7f907406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.549+0000 7f9095b9b700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f907406ace0 0x7f907406d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90900cc810 tx=0x7f908c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.553+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9088055f60 con 0x7f90900cfd10 2023-07-06T21:41:22.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.747+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f909004e1d0 con 0x7f90900cfd10 2023-07-06T21:41:22.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.748+0000 7f90867fc700 1 -- 172.21.15.84:0/2657433216 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9088059580 con 0x7f90900cfd10 2023-07-06T21:41:22.750 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f907406ace0 msgr2=0x7f907406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f907406ace0 0x7f907406d190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90900cc810 tx=0x7f908c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 msgr2=0x7f909015a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 0x7f909015a750 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f909006bed0 tx=0x7f9088004ad0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 shutdown_connections 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f907406ace0 0x7f907406d190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90900cfd10 0x7f909015a750 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90900b1a80 0x7f909015aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 --2- 172.21.15.84:0/2657433216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90900b81c0 0x7f90901e2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 >> 172.21.15.84:0/2657433216 conn(0x7f909006b2b0 msgr2=0x7f90900c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:22.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.750+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 shutdown_connections 2023-07-06T21:41:22.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.751+0000 7f9097dff700 1 -- 172.21.15.84:0/2657433216 wait complete. 2023-07-06T21:41:22.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:22.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.964+0000 7f06013ff700 1 Processor -- start 2023-07-06T21:41:22.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.964+0000 7f06013ff700 1 -- start start 2023-07-06T21:41:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f06013ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b81c0 0x7f05fc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f06013ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0d1830 con 0x7f05fc0b1a80 2023-07-06T21:41:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f06013ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc160d40 con 0x7f05fc0ceda0 2023-07-06T21:41:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f06013ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cf330 con 0x7f05fc0b81c0 2023-07-06T21:41:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f05fa7fc700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f05fc0b1a80 0x7f05fc15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43648/0 (socket says 172.21.15.84:43648) 2023-07-06T21:41:22.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f05fa7fc700 1 -- 172.21.15.84:0/173154367 learned_addr learned my addr 172.21.15.84:0/173154367 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.965+0000 7f05faffd700 1 --2- 172.21.15.84:0/173154367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b81c0 0x7f05fc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843932324 0 0) 0x7f05fc0d1830 con 0x7f05fc0b1a80 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e40035b0 con 0x7f05fc0b1a80 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618759590 0 0) 0x7f05fc160d40 con 0x7f05fc0ceda0 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc0d1830 con 0x7f05fc0ceda0 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05faffd700 1 -- 172.21.15.84:0/173154367 >> v1:172.21.15.107:6789/0 conn(0x7f05fc0ceda0 legacy=0x7f05fc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05faffd700 1 -- 172.21.15.84:0/173154367 >> v1:172.21.15.84:6789/0 conn(0x7f05fc0b1a80 legacy=0x7f05fc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05faffd700 1 -- 172.21.15.84:0/173154367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0d1270 con 0x7f05fc0b81c0 2023-07-06T21:41:22.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05faffd700 1 --2- 172.21.15.84:0/173154367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b81c0 0x7f05fc0cf600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009870 tx=0x7f05ec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51cd840dcffcb08e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.966+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011830 con 0x7f05fc0b81c0 2023-07-06T21:41:22.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec011990 con 0x7f05fc0b81c0 2023-07-06T21:41:22.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f05f9ffb700 1 -- 172.21.15.84:0/173154367 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec011d20 con 0x7f05fc0b81c0 2023-07-06T21:41:22.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 -- 172.21.15.84:0/173154367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b81c0 msgr2=0x7f05fc0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 --2- 172.21.15.84:0/173154367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b81c0 0x7f05fc0cf600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009870 tx=0x7f05ec009b80 comp rx=0 tx=0).stop 2023-07-06T21:41:22.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 -- 172.21.15.84:0/173154367 shutdown_connections 2023-07-06T21:41:22.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 --2- 172.21.15.84:0/173154367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b81c0 0x7f05fc0cf600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 -- 172.21.15.84:0/173154367 >> 172.21.15.84:0/173154367 conn(0x7f05fc06b2b0 msgr2=0x7f05fc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:22.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 -- 172.21.15.84:0/173154367 shutdown_connections 2023-07-06T21:41:22.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.967+0000 7f06013ff700 1 -- 172.21.15.84:0/173154367 wait complete. 2023-07-06T21:41:22.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.969+0000 7f06013ff700 1 Processor -- start 2023-07-06T21:41:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.969+0000 7f06013ff700 1 -- start start 2023-07-06T21:41:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f06013ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b1a80 0x7f05fc0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f06013ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0b81c0 0x7f05fc144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f06013ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 0x7f05fc0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f06013ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0cb850 con 0x7f05fc0ceda0 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f06013ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc155710 con 0x7f05fc0b81c0 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f06013ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc158c70 con 0x7f05fc0b1a80 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f05fb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 0x7f05fc0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f05fb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 0x7f05fc0cbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49428/0 (socket says 172.21.15.84:49428) 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f05fb7fe700 1 -- 172.21.15.84:0/3196498110 learned_addr learned my addr 172.21.15.84:0/3196498110 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:22.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f05faffd700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b1a80 0x7f05fc0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.970+0000 7f05fa7fc700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0b81c0 0x7f05fc144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05fb7fe700 1 -- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b1a80 msgr2=0x7f05fc0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05fb7fe700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b1a80 0x7f05fc0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05fb7fe700 1 -- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0b81c0 msgr2=0x7f05fc144da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05fb7fe700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0b81c0 0x7f05fc144da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05fb7fe700 1 -- 172.21.15.84:0/3196498110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ec0094a0 con 0x7f05fc0ceda0 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05fb7fe700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 0x7f05fc0cbca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f05f400cbc0 tx=0x7f05f400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4003c00 con 0x7f05fc0ceda0 2023-07-06T21:41:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4003d60 con 0x7f05fc0ceda0 2023-07-06T21:41:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f400aaa0 con 0x7f05fc0ceda0 2023-07-06T21:41:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc155a20 con 0x7f05fc0ceda0 2023-07-06T21:41:22.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.971+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc155fe0 con 0x7f05fc0ceda0 2023-07-06T21:41:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.972+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc156170 con 0x7f05fc0ceda0 2023-07-06T21:41:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.973+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f05f400ac00 con 0x7f05fc0ceda0 2023-07-06T21:41:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.974+0000 7f05e37fe700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f05e806ad10 0x7f05e806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.974+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f05f4089640 con 0x7f05fc0ceda0 2023-07-06T21:41:22.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.975+0000 7f05faffd700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f05e806ad10 0x7f05e806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:22.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.975+0000 7f05faffd700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f05e806ad10 0x7f05e806d1c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0097c0 tx=0x7f05ec01c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:22.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:22.978+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f05f40544b0 con 0x7f05fc0ceda0 2023-07-06T21:41:23.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.174+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f05fc0cb990 con 0x7f05fc0ceda0 2023-07-06T21:41:23.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.174+0000 7f05e37fe700 1 -- 172.21.15.84:0/3196498110 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05f4057ad0 con 0x7f05fc0ceda0 2023-07-06T21:41:23.177 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f05e806ad10 msgr2=0x7f05e806d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f05e806ad10 0x7f05e806d1c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f05ec0097c0 tx=0x7f05ec01c040 comp rx=0 tx=0).stop 2023-07-06T21:41:23.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 msgr2=0x7f05fc0cbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 0x7f05fc0cbca0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f05f400cbc0 tx=0x7f05f400ced0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 shutdown_connections 2023-07-06T21:41:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f05e806ad10 0x7f05e806d1c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f05fc0ceda0 0x7f05fc0cbca0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05fc0b1a80 0x7f05fc0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 --2- 172.21.15.84:0/3196498110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05fc0b81c0 0x7f05fc144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 >> 172.21.15.84:0/3196498110 conn(0x7f05fc06b2b0 msgr2=0x7f05fc0715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:23.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 shutdown_connections 2023-07-06T21:41:23.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.177+0000 7f06013ff700 1 -- 172.21.15.84:0/3196498110 wait complete. 2023-07-06T21:41:23.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:23.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.391+0000 7ff7659ff700 1 Processor -- start 2023-07-06T21:41:23.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.391+0000 7ff7659ff700 1 -- start start 2023-07-06T21:41:23.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.393+0000 7ff7659ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff7600cfd10 0x7ff7600d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.393+0000 7ff7659ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600d1830 con 0x7ff7600b1a80 2023-07-06T21:41:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.393+0000 7ff7659ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600cf330 con 0x7ff7600b81c0 2023-07-06T21:41:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.393+0000 7ff7659ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7600d1270 con 0x7ff7600cfd10 2023-07-06T21:41:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.393+0000 7ff75f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff7600b1a80 0x7ff7600d28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43656/0 (socket says 172.21.15.84:43656) 2023-07-06T21:41:23.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75f7fe700 1 -- 172.21.15.84:0/1585131880 learned_addr learned my addr 172.21.15.84:0/1585131880 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:23.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff7567fc700 1 --2- 172.21.15.84:0/1585131880 >> v2:172.21.15.107:3300/0 conn(0x7ff7600cfd10 0x7ff7600d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327445923 0 0) 0x7ff7600d1830 con 0x7ff7600b1a80 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c0035b0 con 0x7ff7600b1a80 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4165527732 0 0) 0x7ff7600cf330 con 0x7ff7600b81c0 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7600d1830 con 0x7ff7600b81c0 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980173617 0 0) 0x7ff74c0035b0 con 0x7ff7600b1a80 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 >> v2:172.21.15.107:3300/0 conn(0x7ff7600cfd10 msgr2=0x7ff7600d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:23.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 --2- 172.21.15.84:0/1585131880 >> v2:172.21.15.107:3300/0 conn(0x7ff7600cfd10 0x7ff7600d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 >> v1:172.21.15.107:6789/0 conn(0x7ff7600b81c0 legacy=0x7ff7600cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600d1550 con 0x7ff7600b1a80 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7ff758001670 con 0x7ff7600b1a80 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 >> v1:172.21.15.84:6789/0 conn(0x7ff7600b1a80 legacy=0x7ff7600d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff74c003b90 0x7ff74c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c004850 0x7ff74c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c005200 0x7ff74c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600d1270 con 0x7ff74c004850 2023-07-06T21:41:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c00c060 con 0x7ff74c005200 2023-07-06T21:41:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.394+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c00c220 con 0x7ff74c003b90 2023-07-06T21:41:23.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c004850 0x7ff74c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff75effd700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff74c003b90 0x7ff74c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff75f7fe700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c005200 0x7ff74c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 -- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff74c003b90 msgr2=0x7ff74c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff74c003b90 0x7ff74c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 -- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c005200 msgr2=0x7ff74c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:23.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c005200 0x7ff74c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 -- 172.21.15.84:0/1585131880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c0035b0 con 0x7ff74c004850 2023-07-06T21:41:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff75f7fe700 1 -- 172.21.15.84:0/1585131880 reap_dead start 2023-07-06T21:41:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.395+0000 7ff7567fc700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c004850 0x7ff74c004cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff748009f90 tx=0x7ff74c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6d5f930387a36b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800ec30 con 0x7ff74c004850 2023-07-06T21:41:23.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74800f040 con 0x7ff74c004850 2023-07-06T21:41:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff75e7fc700 1 -- 172.21.15.84:0/1585131880 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748014480 con 0x7ff74c004850 2023-07-06T21:41:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 -- 172.21.15.84:0/1585131880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c004850 msgr2=0x7ff74c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c004850 0x7ff74c004cc0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff748009f90 tx=0x7ff74c005880 comp rx=0 tx=0).stop 2023-07-06T21:41:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 -- 172.21.15.84:0/1585131880 shutdown_connections 2023-07-06T21:41:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1585131880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff74c004850 0x7ff74c004cc0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 -- 172.21.15.84:0/1585131880 >> 172.21.15.84:0/1585131880 conn(0x7ff76006b2b0 msgr2=0x7ff76006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 -- 172.21.15.84:0/1585131880 shutdown_connections 2023-07-06T21:41:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.396+0000 7ff7659ff700 1 -- 172.21.15.84:0/1585131880 wait complete. 2023-07-06T21:41:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.399+0000 7ff7659ff700 1 Processor -- start 2023-07-06T21:41:23.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.399+0000 7ff7659ff700 1 -- start start 2023-07-06T21:41:23.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff7659ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c004850 0x7ff76015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff7659ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff76014d0c0 0x7ff76015f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff7659ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 0x7ff76015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff7659ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600c8820 con 0x7ff7601f2130 2023-07-06T21:41:23.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff7659ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76014d530 con 0x7ff74c004850 2023-07-06T21:41:23.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff7659ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600c7c10 con 0x7ff76014d0c0 2023-07-06T21:41:23.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff75f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 0x7ff76015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff75f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 0x7ff76015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49434/0 (socket says 172.21.15.84:49434) 2023-07-06T21:41:23.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.400+0000 7ff75f7fe700 1 -- 172.21.15.84:0/1677439262 learned_addr learned my addr 172.21.15.84:0/1677439262 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:23.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff7567fc700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff76014d0c0 0x7ff76015f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75effd700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c004850 0x7ff76015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75f7fe700 1 -- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff76014d0c0 msgr2=0x7ff76015f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75f7fe700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff76014d0c0 0x7ff76015f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75f7fe700 1 -- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c004850 msgr2=0x7ff76015ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75f7fe700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c004850 0x7ff76015ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75f7fe700 1 -- 172.21.15.84:0/1677439262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600c7ef0 con 0x7ff7601f2130 2023-07-06T21:41:23.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75f7fe700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 0x7ff76015a750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758007570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:23.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011580 con 0x7ff7601f2130 2023-07-06T21:41:23.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758011700 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011ab0 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7600c81d0 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.401+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76015fb90 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.402+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff76004e1d0 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.403+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff758011c10 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.405+0000 7ff75cff9700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff74006ace0 0x7ff74006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.405+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7ff758089630 con 0x7ff7601f2130 2023-07-06T21:41:23.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.405+0000 7ff75effd700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff74006ace0 0x7ff74006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.406+0000 7ff75effd700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff74006ace0 0x7ff74006d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff760068f30 tx=0x7ff750009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:23.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.409+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7580544b0 con 0x7ff7601f2130 2023-07-06T21:41:23.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.605+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7ff7600b4f30 con 0x7ff7601f2130 2023-07-06T21:41:23.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.605+0000 7ff75cff9700 1 -- 172.21.15.84:0/1677439262 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7ff758057ad0 con 0x7ff7601f2130 2023-07-06T21:41:23.607 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:23.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff74006ace0 msgr2=0x7ff74006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff74006ace0 0x7ff74006d190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff760068f30 tx=0x7ff750009040 comp rx=0 tx=0).stop 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 msgr2=0x7ff76015a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 0x7ff76015a750 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c00 tx=0x7ff758007570 comp rx=0 tx=0).stop 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 shutdown_connections 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff74006ace0 0x7ff74006d190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7601f2130 0x7ff76015a750 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff74c004850 0x7ff76015ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 --2- 172.21.15.84:0/1677439262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff76014d0c0 0x7ff76015f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 >> 172.21.15.84:0/1677439262 conn(0x7ff76006b2b0 msgr2=0x7ff7600d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:23.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 shutdown_connections 2023-07-06T21:41:23.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.607+0000 7ff7659ff700 1 -- 172.21.15.84:0/1677439262 wait complete. 2023-07-06T21:41:23.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:41:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.819+0000 7f7a103ff700 1 Processor -- start 2023-07-06T21:41:23.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.819+0000 7f7a103ff700 1 -- start start 2023-07-06T21:41:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a103ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7a080d3580 0x7f7a080d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a103ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a080d13b0 0x7f7a0815c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a103ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a080d0990 con 0x7f7a080b8530 2023-07-06T21:41:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a103ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a080d3af0 con 0x7f7a080d13b0 2023-07-06T21:41:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a103ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a080cd9d0 con 0x7f7a080d3580 2023-07-06T21:41:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a0e99c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a080d13b0 0x7f7a0815c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.820+0000 7f7a0e99c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7a080d13b0 0x7f7a0815c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49438/0 (socket says 172.21.15.84:49438) 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0e99c700 1 -- 172.21.15.84:0/915189013 learned_addr learned my addr 172.21.15.84:0/915189013 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0d99a700 1 --2- 172.21.15.84:0/915189013 >> v2:172.21.15.107:3300/0 conn(0x7f7a080d3580 0x7f7a080d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0e99c700 1 -- 172.21.15.84:0/915189013 >> v2:172.21.15.107:3300/0 conn(0x7f7a080d3580 msgr2=0x7f7a080d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0e99c700 1 --2- 172.21.15.84:0/915189013 >> v2:172.21.15.107:3300/0 conn(0x7f7a080d3580 0x7f7a080d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0e99c700 1 -- 172.21.15.84:0/915189013 >> v1:172.21.15.107:6789/0 conn(0x7f7a080b8530 legacy=0x7f7a080cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0e99c700 1 -- 172.21.15.84:0/915189013 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a080cd410 con 0x7f7a080d13b0 2023-07-06T21:41:23.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0e99c700 1 --2- 172.21.15.84:0/915189013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a0815c8e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7a080d07a0 tx=0x7f7a0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e38d06a2f13d379 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0d199700 1 -- 172.21.15.84:0/915189013 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000e070 con 0x7f7a080d13b0 2023-07-06T21:41:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0d199700 1 -- 172.21.15.84:0/915189013 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0000b9c0 con 0x7f7a080d13b0 2023-07-06T21:41:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.821+0000 7f7a0d199700 1 -- 172.21.15.84:0/915189013 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0000bd70 con 0x7f7a080d13b0 2023-07-06T21:41:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 -- 172.21.15.84:0/915189013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 msgr2=0x7f7a0815c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 --2- 172.21.15.84:0/915189013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a0815c8e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7a080d07a0 tx=0x7f7a0000b330 comp rx=0 tx=0).stop 2023-07-06T21:41:23.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 -- 172.21.15.84:0/915189013 shutdown_connections 2023-07-06T21:41:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 --2- 172.21.15.84:0/915189013 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a0815c8e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 --2- 172.21.15.84:0/915189013 >> v2:172.21.15.107:3300/0 conn(0x7f7a080d3580 0x7f7a080d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 -- 172.21.15.84:0/915189013 >> 172.21.15.84:0/915189013 conn(0x7f7a080715c0 msgr2=0x7f7a08071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 -- 172.21.15.84:0/915189013 shutdown_connections 2023-07-06T21:41:23.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.822+0000 7f7a103ff700 1 -- 172.21.15.84:0/915189013 wait complete. 2023-07-06T21:41:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.824+0000 7f7a103ff700 1 Processor -- start 2023-07-06T21:41:23.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.824+0000 7f7a103ff700 1 -- start start 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a103ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a080b8530 0x7f7a081540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a103ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a080c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a103ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a080d3580 0x7f7a08158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a103ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080c9320 con 0x7f7a080d13b0 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a103ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080c86a0 con 0x7f7a080b8530 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a103ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a080c8980 con 0x7f7a080d3580 2023-07-06T21:41:23.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a0d99a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a080c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a0d99a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a080c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49448/0 (socket says 172.21.15.84:49448) 2023-07-06T21:41:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a0d99a700 1 -- 172.21.15.84:0/3132630924 learned_addr learned my addr 172.21.15.84:0/3132630924 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a0e99c700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a080d3580 0x7f7a08158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.825+0000 7f7a0e19b700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a080b8530 0x7f7a081540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a0d99a700 1 -- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a080d3580 msgr2=0x7f7a08158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a0d99a700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a080d3580 0x7f7a08158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a0d99a700 1 -- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a080b8530 msgr2=0x7f7a081540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a0d99a700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a080b8530 0x7f7a081540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a0d99a700 1 -- 172.21.15.84:0/3132630924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a00009b30 con 0x7f7a080d13b0 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a0d99a700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a080c7cc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f79f800d800 tx=0x7f79f800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80125f0 con 0x7f7a080d13b0 2023-07-06T21:41:23.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8012750 con 0x7f7a080d13b0 2023-07-06T21:41:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a080c8c60 con 0x7f7a080d13b0 2023-07-06T21:41:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8012b00 con 0x7f7a080d13b0 2023-07-06T21:41:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.826+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a081545f0 con 0x7f7a080d13b0 2023-07-06T21:41:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.827+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0804e240 con 0x7f7a080d13b0 2023-07-06T21:41:23.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.828+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f79f8012c60 con 0x7f7a080d13b0 2023-07-06T21:41:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.829+0000 7f79feffd700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f79ec06ace0 0x7f79ec06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.829+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f79f8089a70 con 0x7f7a080d13b0 2023-07-06T21:41:23.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.830+0000 7f7a0e19b700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f79ec06ace0 0x7f79ec06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:23.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.830+0000 7f7a0e19b700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f79ec06ace0 0x7f79ec06d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000fb0 tx=0x7f7a04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:23.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:23.834+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f79f80548f0 con 0x7f7a080d13b0 2023-07-06T21:41:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.047+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7a080b47f0 con 0x7f7a080d13b0 2023-07-06T21:41:24.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.048+0000 7f79feffd700 1 -- 172.21.15.84:0/3132630924 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79f8057f10 con 0x7f7a080d13b0 2023-07-06T21:41:24.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f79ec06ace0 msgr2=0x7f79ec06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f79ec06ace0 0x7f79ec06d190 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a04000fb0 tx=0x7f7a04009040 comp rx=0 tx=0).stop 2023-07-06T21:41:24.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 msgr2=0x7f7a080c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a080c7cc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f79f800d800 tx=0x7f79f800dbc0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 shutdown_connections 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f79ec06ace0 0x7f79ec06d190 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a080d3580 0x7f7a08158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a080b8530 0x7f7a081540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 --2- 172.21.15.84:0/3132630924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a080d13b0 0x7f7a080c7cc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 >> 172.21.15.84:0/3132630924 conn(0x7f7a080715c0 msgr2=0x7f7a080cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 shutdown_connections 2023-07-06T21:41:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.050+0000 7f7a103ff700 1 -- 172.21.15.84:0/3132630924 wait complete. 2023-07-06T21:41:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:41:24.070 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:24.071 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:24.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.348+0000 7fdbdb5ff700 1 Processor -- start 2023-07-06T21:41:24.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.348+0000 7fdbdb5ff700 1 -- start start 2023-07-06T21:41:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.349+0000 7fdbdb5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd4070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.349+0000 7fdbdb5ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdbd40c9c10 0x7fdbd40ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.349+0000 7fdbdb5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd415d930 con 0x7fdbd4159130 2023-07-06T21:41:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.349+0000 7fdbdb5ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd4070bc0 con 0x7fdbd40c9c10 2023-07-06T21:41:24.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.349+0000 7fdbdb5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd415cd20 con 0x7fdbd4070230 2023-07-06T21:41:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd8b9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd4070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd939b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fdbd4159130 0x7fdbd40c4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:45544/0 (socket says 172.21.15.84:45544) 2023-07-06T21:41:24.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd9b9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdbd40c9c10 0x7fdbd40ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd939b700 1 -- 172.21.15.84:0/3109737630 learned_addr learned my addr 172.21.15.84:0/3109737630 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbcbfff700 1 -- 172.21.15.84:0/3109737630 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629781026 0 0) 0x7fdbd415d930 con 0x7fdbd4159130 2023-07-06T21:41:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbcbfff700 1 -- 172.21.15.84:0/3109737630 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbc00057c0 con 0x7fdbd4159130 2023-07-06T21:41:24.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/3109737630 >> v2:172.21.15.107:3300/0 conn(0x7fdbd40c9c10 msgr2=0x7fdbd40ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd8b9a700 1 --2- 172.21.15.84:0/3109737630 >> v2:172.21.15.107:3300/0 conn(0x7fdbd40c9c10 0x7fdbd40ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/3109737630 >> v1:172.21.15.107:6789/0 conn(0x7fdbd4159130 legacy=0x7fdbd40c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/3109737630 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd415d000 con 0x7fdbd4070230 2023-07-06T21:41:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd9b9c700 1 --2- 172.21.15.84:0/3109737630 >> v2:172.21.15.107:3300/0 conn(0x7fdbd40c9c10 0x7fdbd40ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.350+0000 7fdbd8b9a700 1 --2- 172.21.15.84:0/3109737630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd4070680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009840 tx=0x7fdbc4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58acef03a04d84b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbcbfff700 1 -- 172.21.15.84:0/3109737630 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4011830 con 0x7fdbd4070230 2023-07-06T21:41:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbcbfff700 1 -- 172.21.15.84:0/3109737630 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4011990 con 0x7fdbd4070230 2023-07-06T21:41:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbcbfff700 1 -- 172.21.15.84:0/3109737630 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4011d20 con 0x7fdbd4070230 2023-07-06T21:41:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/3109737630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 msgr2=0x7fdbd4070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/3109737630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd4070680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4009840 tx=0x7fdbc4009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/3109737630 shutdown_connections 2023-07-06T21:41:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/3109737630 >> v2:172.21.15.107:3300/0 conn(0x7fdbd40c9c10 0x7fdbd40ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/3109737630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd4070680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/3109737630 >> 172.21.15.84:0/3109737630 conn(0x7fdbd40cf3d0 msgr2=0x7fdbd40cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/3109737630 shutdown_connections 2023-07-06T21:41:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.351+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/3109737630 wait complete. 2023-07-06T21:41:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.353+0000 7fdbdb5ff700 1 Processor -- start 2023-07-06T21:41:24.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.353+0000 7fdbdb5ff700 1 -- start start 2023-07-06T21:41:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbdb5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbdb5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 0x7fdbd414ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbdb5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd4159130 0x7fdbd40af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbdb5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd414c410 con 0x7fdbd40c9c10 2023-07-06T21:41:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbdb5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd414c6f0 con 0x7fdbd4159130 2023-07-06T21:41:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbdb5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd414c9d0 con 0x7fdbd4070230 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbd8b9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 0x7fdbd414ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbd8b9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 0x7fdbd414ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49462/0 (socket says 172.21.15.84:49462) 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.354+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/2620483858 learned_addr learned my addr 172.21.15.84:0/2620483858 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd9b9c700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd4159130 0x7fdbd40af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd939b700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 msgr2=0x7fdbd41f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd8b9a700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd41f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd4159130 msgr2=0x7fdbd40af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd8b9a700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd4159130 0x7fdbd40af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd8b9a700 1 -- 172.21.15.84:0/2620483858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc009a70 con 0x7fdbd40c9c10 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbd8b9a700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 0x7fdbd414ba50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40090b0 tx=0x7fdbc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc40036a0 con 0x7fdbd40c9c10 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc401f070 con 0x7fdbd40c9c10 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4003a40 con 0x7fdbd40c9c10 2023-07-06T21:41:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.355+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc40094a0 con 0x7fdbd40c9c10 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.356+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd414cf90 con 0x7fdbd40c9c10 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.357+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd40ce0f0 con 0x7fdbd40c9c10 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.357+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdbc4003ba0 con 0x7fdbd40c9c10 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.358+0000 7fdbc9ffb700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdbb806ace0 0x7fdbb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.358+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdbc4021070 con 0x7fdbd40c9c10 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.359+0000 7fdbd939b700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdbb806ace0 0x7fdbb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.359+0000 7fdbd939b700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdbb806ace0 0x7fdbb806d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00011e0 tx=0x7fdbd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.363+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdbc4055e10 con 0x7fdbd40c9c10 2023-07-06T21:41:24.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.561+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd40ad820 con 0x7fdbd40c9c10 2023-07-06T21:41:24.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.562+0000 7fdbc9ffb700 1 -- 172.21.15.84:0/2620483858 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7fdbc4059430 con 0x7fdbd40c9c10 2023-07-06T21:41:24.565 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:24.565 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdbb806ace0 msgr2=0x7fdbb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdbb806ace0 0x7fdbb806d190 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00011e0 tx=0x7fdbd0009040 comp rx=0 tx=0).stop 2023-07-06T21:41:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 msgr2=0x7fdbd414ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 0x7fdbd414ba50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40090b0 tx=0x7fdbc4004750 comp rx=0 tx=0).stop 2023-07-06T21:41:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 shutdown_connections 2023-07-06T21:41:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdbb806ace0 0x7fdbb806d190 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbd4159130 0x7fdbd40af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbd4070230 0x7fdbd41f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 --2- 172.21.15.84:0/2620483858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdbd40c9c10 0x7fdbd414ba50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.565+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 >> 172.21.15.84:0/2620483858 conn(0x7fdbd40cf3d0 msgr2=0x7fdbd40ca030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.566+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 shutdown_connections 2023-07-06T21:41:24.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.566+0000 7fdbdb5ff700 1 -- 172.21.15.84:0/2620483858 wait complete. 2023-07-06T21:41:24.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:24.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.781+0000 7f8ab73ff700 1 Processor -- start 2023-07-06T21:41:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.782+0000 7f8ab73ff700 1 -- start start 2023-07-06T21:41:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab73ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8ab00b81c0 0x7f8ab00cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab73ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8ab00ceda0 0x7f8ab00cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab73ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab00d1830 con 0x7f8ab00b1a80 2023-07-06T21:41:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab73ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab00cf330 con 0x7f8ab00ceda0 2023-07-06T21:41:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab73ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab00d1270 con 0x7f8ab00b81c0 2023-07-06T21:41:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8ab00ceda0 0x7f8ab00cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8ab00ceda0 0x7f8ab00cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49472/0 (socket says 172.21.15.84:49472) 2023-07-06T21:41:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 -- 172.21.15.84:0/1382368818 learned_addr learned my addr 172.21.15.84:0/1382368818 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab519b700 1 --2- 172.21.15.84:0/1382368818 >> v2:172.21.15.107:3300/0 conn(0x7f8ab00b81c0 0x7f8ab00cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 -- 172.21.15.84:0/1382368818 >> v2:172.21.15.107:3300/0 conn(0x7f8ab00b81c0 msgr2=0x7f8ab00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 --2- 172.21.15.84:0/1382368818 >> v2:172.21.15.107:3300/0 conn(0x7f8ab00b81c0 0x7f8ab00cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 -- 172.21.15.84:0/1382368818 >> v1:172.21.15.107:6789/0 conn(0x7f8ab00b1a80 legacy=0x7f8ab015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab599c700 1 -- 172.21.15.84:0/1382368818 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab00d1550 con 0x7f8ab00ceda0 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.783+0000 7f8ab519b700 1 --2- 172.21.15.84:0/1382368818 >> v2:172.21.15.107:3300/0 conn(0x7f8ab00b81c0 0x7f8ab00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8ab599c700 1 --2- 172.21.15.84:0/1382368818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00ceda0 0x7f8ab00cfb40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00b2620 tx=0x7f8aa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fe92ca4cb6d3db server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8aa7fff700 1 -- 172.21.15.84:0/1382368818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800e070 con 0x7f8ab00ceda0 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8aa7fff700 1 -- 172.21.15.84:0/1382368818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa800b9c0 con 0x7f8ab00ceda0 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8aa7fff700 1 -- 172.21.15.84:0/1382368818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800bd70 con 0x7f8ab00ceda0 2023-07-06T21:41:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8ab73ff700 1 -- 172.21.15.84:0/1382368818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00ceda0 msgr2=0x7f8ab00cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/1382368818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00ceda0 0x7f8ab00cfb40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00b2620 tx=0x7f8aa800b330 comp rx=0 tx=0).stop 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.784+0000 7f8ab73ff700 1 -- 172.21.15.84:0/1382368818 shutdown_connections 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.785+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/1382368818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00ceda0 0x7f8ab00cfb40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.785+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/1382368818 >> v2:172.21.15.107:3300/0 conn(0x7f8ab00b81c0 0x7f8ab00cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.785+0000 7f8ab73ff700 1 -- 172.21.15.84:0/1382368818 >> 172.21.15.84:0/1382368818 conn(0x7f8ab006b2b0 msgr2=0x7f8ab006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.785+0000 7f8ab73ff700 1 -- 172.21.15.84:0/1382368818 shutdown_connections 2023-07-06T21:41:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.785+0000 7f8ab73ff700 1 -- 172.21.15.84:0/1382368818 wait complete. 2023-07-06T21:41:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.786+0000 7f8ab73ff700 1 Processor -- start 2023-07-06T21:41:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.786+0000 7f8ab73ff700 1 -- start start 2023-07-06T21:41:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.787+0000 7f8ab73ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 0x7f8ab00cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.787+0000 7f8ab73ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab00b81c0 0x7f8ab01f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.787+0000 7f8ab73ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ab00ceda0 0x7f8ab01564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.787+0000 7f8ab73ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab0156a20 con 0x7f8ab00b1a80 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.787+0000 7f8ab73ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01f2cb0 con 0x7f8ab00b81c0 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.787+0000 7f8ab73ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab01f2f00 con 0x7f8ab00ceda0 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 0x7f8ab00cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 0x7f8ab00cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49488/0 (socket says 172.21.15.84:49488) 2023-07-06T21:41:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 -- 172.21.15.84:0/3352660131 learned_addr learned my addr 172.21.15.84:0/3352660131 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab499a700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab00b81c0 0x7f8ab01f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab599c700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ab00ceda0 0x7f8ab01564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 -- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ab00ceda0 msgr2=0x7f8ab01564e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ab00ceda0 0x7f8ab01564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 -- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab00b81c0 msgr2=0x7f8ab01f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab00b81c0 0x7f8ab01f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 -- 172.21.15.84:0/3352660131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8009b30 con 0x7f8ab00b1a80 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.788+0000 7f8ab519b700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 0x7f8ab00cc9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00e810 tx=0x7f8aac00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.789+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac013610 con 0x7f8ab00b1a80 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.789+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac013770 con 0x7f8ab00b1a80 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.789+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab01f31e0 con 0x7f8ab00b1a80 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.789+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac013b20 con 0x7f8ab00b1a80 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.789+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab01f37a0 con 0x7f8ab00b1a80 2023-07-06T21:41:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.790+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab00d3370 con 0x7f8ab00b1a80 2023-07-06T21:41:24.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.791+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8aac013c80 con 0x7f8ab00b1a80 2023-07-06T21:41:24.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.792+0000 7f8aa5ffb700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a9406ace0 0x7f8a9406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.792+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8aac089150 con 0x7f8ab00b1a80 2023-07-06T21:41:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.792+0000 7f8ab499a700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a9406ace0 0x7f8a9406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.792+0000 7f8ab499a700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a9406ace0 0x7f8a9406d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00b5dd0 tx=0x7f8aa0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:24.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.796+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8ab00d3370 con 0x7f8ab00b1a80 2023-07-06T21:41:24.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.992+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ab00d3350 con 0x7f8ab00b1a80 2023-07-06T21:41:24.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.992+0000 7f8aa5ffb700 1 -- 172.21.15.84:0/3352660131 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8aac057b90 con 0x7f8ab00b1a80 2023-07-06T21:41:24.995 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:24.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a9406ace0 msgr2=0x7f8a9406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a9406ace0 0x7f8a9406d190 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00b5dd0 tx=0x7f8aa0009040 comp rx=0 tx=0).stop 2023-07-06T21:41:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 msgr2=0x7f8ab00cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 0x7f8ab00cc9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00e810 tx=0x7f8aac00eb20 comp rx=0 tx=0).stop 2023-07-06T21:41:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 shutdown_connections 2023-07-06T21:41:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8a9406ace0 0x7f8a9406d190 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ab00ceda0 0x7f8ab01564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab00b1a80 0x7f8ab00cc9a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 --2- 172.21.15.84:0/3352660131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ab00b81c0 0x7f8ab01f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:24.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 >> 172.21.15.84:0/3352660131 conn(0x7f8ab006b2b0 msgr2=0x7f8ab00cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:24.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 shutdown_connections 2023-07-06T21:41:24.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:24.995+0000 7f8ab73ff700 1 -- 172.21.15.84:0/3352660131 wait complete. 2023-07-06T21:41:25.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:25.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.212+0000 7fee72bff700 1 Processor -- start 2023-07-06T21:41:25.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.213+0000 7fee72bff700 1 -- start start 2023-07-06T21:41:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee72bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 0x7fee6c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee72bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c0d1830 con 0x7fee6c0b1a80 2023-07-06T21:41:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee72bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6c160d40 con 0x7fee6c0ceda0 2023-07-06T21:41:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee72bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0cf330 con 0x7fee6c0b81c0 2023-07-06T21:41:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee6bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fee6c0b1a80 0x7fee6c15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43668/0 (socket says 172.21.15.84:43668) 2023-07-06T21:41:25.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee6bfff700 1 -- 172.21.15.84:0/3894187642 learned_addr learned my addr 172.21.15.84:0/3894187642 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee7099b700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 0x7fee6c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.214+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170347357 0 0) 0x7fee6c0d1830 con 0x7fee6c0b1a80 2023-07-06T21:41:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee5c0035b0 con 0x7fee6c0b1a80 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660564566 0 0) 0x7fee6c160d40 con 0x7fee6c0ceda0 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee6c0d1830 con 0x7fee6c0ceda0 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1573258254 0 0) 0x7fee5c0035b0 con 0x7fee6c0b1a80 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 msgr2=0x7fee6c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 0x7fee6c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 >> v1:172.21.15.107:6789/0 conn(0x7fee6c0ceda0 legacy=0x7fee6c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0d1270 con 0x7fee6c0b1a80 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fee58001670 con 0x7fee6c0b1a80 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 >> v1:172.21.15.84:6789/0 conn(0x7fee6c0b1a80 legacy=0x7fee6c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c003bc0 0x7fee5c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 0x7fee5c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5c004d90 0x7fee5c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0cf330 con 0x7fee5c004d90 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c00bac0 con 0x7fee5c003bc0 2023-07-06T21:41:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c00bc80 con 0x7fee6c0b81c0 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee7119c700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5c004d90 0x7fee5c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee6bfff700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 0x7fee5c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.215+0000 7fee7099b700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c003bc0 0x7fee5c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee7119c700 1 -- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 msgr2=0x7fee5c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee7119c700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c0b81c0 0x7fee5c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee7119c700 1 -- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c003bc0 msgr2=0x7fee5c003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee7119c700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c003bc0 0x7fee5c003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee7119c700 1 -- 172.21.15.84:0/3894187642 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0035b0 con 0x7fee5c004d90 2023-07-06T21:41:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee7119c700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5c004d90 0x7fee5c005200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fee64002230 tx=0x7fee5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe185e696b5f1dc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64001f30 con 0x7fee5c004d90 2023-07-06T21:41:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6400ce80 con 0x7fee5c004d90 2023-07-06T21:41:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.216+0000 7fee6b7fe700 1 -- 172.21.15.84:0/3894187642 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64003c40 con 0x7fee5c004d90 2023-07-06T21:41:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee72bff700 1 -- 172.21.15.84:0/3894187642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5c004d90 msgr2=0x7fee5c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee72bff700 1 --2- 172.21.15.84:0/3894187642 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee5c004d90 0x7fee5c005200 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fee64002230 tx=0x7fee5c005880 comp rx=0 tx=0).stop 2023-07-06T21:41:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee7119c700 1 -- 172.21.15.84:0/3894187642 reap_dead start 2023-07-06T21:41:25.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee72bff700 1 -- 172.21.15.84:0/3894187642 shutdown_connections 2023-07-06T21:41:25.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee72bff700 1 -- 172.21.15.84:0/3894187642 >> 172.21.15.84:0/3894187642 conn(0x7fee6c06b2b0 msgr2=0x7fee6c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:25.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee72bff700 1 -- 172.21.15.84:0/3894187642 shutdown_connections 2023-07-06T21:41:25.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.217+0000 7fee72bff700 1 -- 172.21.15.84:0/3894187642 wait complete. 2023-07-06T21:41:25.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.219+0000 7fee72bff700 1 Processor -- start 2023-07-06T21:41:25.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.219+0000 7fee72bff700 1 -- start start 2023-07-06T21:41:25.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee72bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c004d90 0x7fee6c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee72bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 0x7fee6c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee72bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c147720 0x7fee6c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee72bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0cb7c0 con 0x7fee6c155f30 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee72bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0cbaa0 con 0x7fee5c004d90 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee72bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6c0cbd80 con 0x7fee6c147720 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 0x7fee6c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 0x7fee6c1471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49506/0 (socket says 172.21.15.84:49506) 2023-07-06T21:41:25.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 -- 172.21.15.84:0/3408586718 learned_addr learned my addr 172.21.15.84:0/3408586718 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee7099b700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c004d90 0x7fee6c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee7119c700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c147720 0x7fee6c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 -- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c147720 msgr2=0x7fee6c154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c147720 0x7fee6c154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 -- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c004d90 msgr2=0x7fee6c1559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c004d90 0x7fee6c1559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee6bfff700 1 -- 172.21.15.84:0/3408586718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6c0cc060 con 0x7fee6c155f30 2023-07-06T21:41:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.220+0000 7fee7099b700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c004d90 0x7fee6c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.221+0000 7fee6bfff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 0x7fee6c1471e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fee58007290 tx=0x7fee580075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.221+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800f6c0 con 0x7fee6c155f30 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.221+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800f860 con 0x7fee6c155f30 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.221+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6c0cc340 con 0x7fee6c155f30 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.221+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5800fc10 con 0x7fee6c155f30 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.221+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6c154ff0 con 0x7fee6c155f30 2023-07-06T21:41:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.222+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee6c147af0 con 0x7fee6c155f30 2023-07-06T21:41:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.223+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fee5800fd70 con 0x7fee6c155f30 2023-07-06T21:41:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.224+0000 7fee697fa700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fee5006ad10 0x7fee5006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.224+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fee580896c0 con 0x7fee6c155f30 2023-07-06T21:41:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.224+0000 7fee7099b700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fee5006ad10 0x7fee5006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.225+0000 7fee7099b700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fee5006ad10 0x7fee5006d1c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee60009c60 tx=0x7fee60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.228+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fee6c147af0 con 0x7fee6c155f30 2023-07-06T21:41:25.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.422+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fee6c147af0 con 0x7fee6c155f30 2023-07-06T21:41:25.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.423+0000 7fee697fa700 1 -- 172.21.15.84:0/3408586718 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fee6c147af0 con 0x7fee6c155f30 2023-07-06T21:41:25.425 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.425+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fee5006ad10 msgr2=0x7fee5006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.425+0000 7fee72bff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fee5006ad10 0x7fee5006d1c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee60009c60 tx=0x7fee60008040 comp rx=0 tx=0).stop 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.425+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 msgr2=0x7fee6c1471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.425+0000 7fee72bff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 0x7fee6c1471e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fee58007290 tx=0x7fee580075a0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 shutdown_connections 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fee5006ad10 0x7fee5006d1c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee6c147720 0x7fee6c154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee5c004d90 0x7fee6c1559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 --2- 172.21.15.84:0/3408586718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fee6c155f30 0x7fee6c1471e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 >> 172.21.15.84:0/3408586718 conn(0x7fee6c06b2b0 msgr2=0x7fee6c0d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:25.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 shutdown_connections 2023-07-06T21:41:25.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.426+0000 7fee72bff700 1 -- 172.21.15.84:0/3408586718 wait complete. 2023-07-06T21:41:25.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:25.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.637+0000 7f370d7ff700 1 Processor -- start 2023-07-06T21:41:25.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.637+0000 7f370d7ff700 1 -- start start 2023-07-06T21:41:25.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f370d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080b81c0 0x7f37080cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f370d7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f37080b1a80 0x7f37080cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f370d7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f37080ceda0 0x7f370815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f370d7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080d1830 con 0x7f37080b1a80 2023-07-06T21:41:25.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f370d7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37080cf330 con 0x7f37080ceda0 2023-07-06T21:41:25.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f370d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37080d1270 con 0x7f37080b81c0 2023-07-06T21:41:25.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.638+0000 7f37067fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f37080b1a80 0x7f37080cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f37080b1a80 0x7f37080cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49512/0 (socket says 172.21.15.84:49512) 2023-07-06T21:41:25.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 -- 172.21.15.84:0/2356249482 learned_addr learned my addr 172.21.15.84:0/2356249482 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:25.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f3706ffd700 1 --2- 172.21.15.84:0/2356249482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080b81c0 0x7f37080cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37077fe700 1 --2- 172.21.15.84:0/2356249482 >> v2:172.21.15.107:3300/0 conn(0x7f37080ceda0 0x7f370815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 -- 172.21.15.84:0/2356249482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080b81c0 msgr2=0x7f37080cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:25.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 --2- 172.21.15.84:0/2356249482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080b81c0 0x7f37080cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 -- 172.21.15.84:0/2356249482 >> v2:172.21.15.107:3300/0 conn(0x7f37080ceda0 msgr2=0x7f370815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:25.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 --2- 172.21.15.84:0/2356249482 >> v2:172.21.15.107:3300/0 conn(0x7f37080ceda0 0x7f370815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 -- 172.21.15.84:0/2356249482 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37080d1550 con 0x7f37080b1a80 2023-07-06T21:41:25.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f37067fc700 1 --2- 172.21.15.84:0/2356249482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b1a80 0x7f37080cfb40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f37080b8d30 tx=0x7f36fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129fe7c2b5d21c20 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:25.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.639+0000 7f3705ffb700 1 -- 172.21.15.84:0/2356249482 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011830 con 0x7f37080b1a80 2023-07-06T21:41:25.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f3705ffb700 1 -- 172.21.15.84:0/2356249482 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc011990 con 0x7f37080b1a80 2023-07-06T21:41:25.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f3705ffb700 1 -- 172.21.15.84:0/2356249482 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc011d20 con 0x7f37080b1a80 2023-07-06T21:41:25.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 -- 172.21.15.84:0/2356249482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b1a80 msgr2=0x7f37080cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 --2- 172.21.15.84:0/2356249482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b1a80 0x7f37080cfb40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f37080b8d30 tx=0x7f36fc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 -- 172.21.15.84:0/2356249482 shutdown_connections 2023-07-06T21:41:25.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 --2- 172.21.15.84:0/2356249482 >> v2:172.21.15.107:3300/0 conn(0x7f37080ceda0 0x7f370815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 --2- 172.21.15.84:0/2356249482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080b81c0 0x7f37080cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 --2- 172.21.15.84:0/2356249482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b1a80 0x7f37080cfb40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 -- 172.21.15.84:0/2356249482 >> 172.21.15.84:0/2356249482 conn(0x7f370806b2b0 msgr2=0x7f370806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:25.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 -- 172.21.15.84:0/2356249482 shutdown_connections 2023-07-06T21:41:25.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.640+0000 7f370d7ff700 1 -- 172.21.15.84:0/2356249482 wait complete. 2023-07-06T21:41:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.642+0000 7f370d7ff700 1 Processor -- start 2023-07-06T21:41:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.642+0000 7f370d7ff700 1 -- start start 2023-07-06T21:41:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f370d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37080b1a80 0x7f37080c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f370d7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 0x7f37081f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f370d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080ceda0 0x7f37081e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f370d7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3708158760 con 0x7f37080b81c0 2023-07-06T21:41:25.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f370d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37080c9890 con 0x7f37080b1a80 2023-07-06T21:41:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f370d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37081f2810 con 0x7f37080ceda0 2023-07-06T21:41:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f37067fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 0x7f37081f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f37067fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 0x7f37081f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49524/0 (socket says 172.21.15.84:49524) 2023-07-06T21:41:25.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.643+0000 7f37067fc700 1 -- 172.21.15.84:0/3113044027 learned_addr learned my addr 172.21.15.84:0/3113044027 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37077fe700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080ceda0 0x7f37081e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f3706ffd700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37080b1a80 0x7f37080c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37067fc700 1 -- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080ceda0 msgr2=0x7f37081e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37067fc700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080ceda0 0x7f37081e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37067fc700 1 -- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37080b1a80 msgr2=0x7f37080c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37067fc700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37080b1a80 0x7f37080c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.659 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37067fc700 1 -- 172.21.15.84:0/3113044027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009550 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f37067fc700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 0x7f37081f22d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000c00 tx=0x7f36fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc004240 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc01f070 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37081e5020 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0036a0 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.644+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37081e59b0 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.645+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37081e5520 con 0x7f37080b81c0 2023-07-06T21:41:25.660 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.646+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f36fc021030 con 0x7f37080b81c0 2023-07-06T21:41:25.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.647+0000 7f36ef7fe700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f406ace0 0x7f36f406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:25.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.647+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f36fc08a870 con 0x7f37080b81c0 2023-07-06T21:41:25.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.647+0000 7f3706ffd700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f406ace0 0x7f36f406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:25.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.648+0000 7f3706ffd700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f406ace0 0x7f36f406d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009be0 tx=0x7f36f8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:25.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.651+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f37081e5520 con 0x7f37080b81c0 2023-07-06T21:41:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.853+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37081e5520 con 0x7f37080b81c0 2023-07-06T21:41:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.853+0000 7f36ef7fe700 1 -- 172.21.15.84:0/3113044027 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37081e5520 con 0x7f37080b81c0 2023-07-06T21:41:25.857 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f406ace0 msgr2=0x7f36f406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f406ace0 0x7f36f406d190 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36f8009be0 tx=0x7f36f8009210 comp rx=0 tx=0).stop 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 msgr2=0x7f37081f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 0x7f37081f22d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000c00 tx=0x7f36fc004750 comp rx=0 tx=0).stop 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 shutdown_connections 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f406ace0 0x7f36f406d190 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37080ceda0 0x7f37081e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37080b1a80 0x7f37080c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 --2- 172.21.15.84:0/3113044027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f37080b81c0 0x7f37081f22d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.855+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 >> 172.21.15.84:0/3113044027 conn(0x7f370806b2b0 msgr2=0x7f37080b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.856+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 shutdown_connections 2023-07-06T21:41:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:25.856+0000 7f370d7ff700 1 -- 172.21.15.84:0/3113044027 wait complete. 2023-07-06T21:41:25.876 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.069+0000 7f6b8cfff700 1 Processor -- start 2023-07-06T21:41:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.069+0000 7f6b8cfff700 1 -- start start 2023-07-06T21:41:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.070+0000 7f6b8cfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880b1a80 0x7f6b880d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.070+0000 7f6b8cfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b880d1830 con 0x7f6b880b81c0 2023-07-06T21:41:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b8cfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b880cf330 con 0x7f6b880cfd10 2023-07-06T21:41:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b8cfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880d1270 con 0x7f6b880b1a80 2023-07-06T21:41:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b8659c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f6b880b81c0 0x7f6b880cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43676/0 (socket says 172.21.15.84:43676) 2023-07-06T21:41:26.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b8659c700 1 -- 172.21.15.84:0/4221902304 learned_addr learned my addr 172.21.15.84:0/4221902304 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:26.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b86d9d700 1 --2- 172.21.15.84:0/4221902304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880b1a80 0x7f6b880d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b85d9b700 1 -- 172.21.15.84:0/4221902304 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645316736 0 0) 0x7f6b880d1830 con 0x7f6b880b81c0 2023-07-06T21:41:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b85d9b700 1 -- 172.21.15.84:0/4221902304 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b740035b0 con 0x7f6b880b81c0 2023-07-06T21:41:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b85d9b700 1 -- 172.21.15.84:0/4221902304 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3637494023 0 0) 0x7f6b740035b0 con 0x7f6b880b81c0 2023-07-06T21:41:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b86d9d700 1 -- 172.21.15.84:0/4221902304 >> v1:172.21.15.107:6789/0 conn(0x7f6b880cfd10 legacy=0x7f6b880d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b86d9d700 1 -- 172.21.15.84:0/4221902304 >> v1:172.21.15.84:6789/0 conn(0x7f6b880b81c0 legacy=0x7f6b880cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.071+0000 7f6b86d9d700 1 -- 172.21.15.84:0/4221902304 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880d1550 con 0x7f6b880b1a80 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.072+0000 7f6b86d9d700 1 --2- 172.21.15.84:0/4221902304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880b1a80 0x7f6b880d2a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009dc0 tx=0x7f6b8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3276ee9a727abf39 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.072+0000 7f6b85d9b700 1 -- 172.21.15.84:0/4221902304 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000e070 con 0x7f6b880b1a80 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.072+0000 7f6b85d9b700 1 -- 172.21.15.84:0/4221902304 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b80003a20 con 0x7f6b880b1a80 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.072+0000 7f6b85d9b700 1 -- 172.21.15.84:0/4221902304 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78001670 con 0x7f6b880b1a80 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.072+0000 7f6b8cfff700 1 -- 172.21.15.84:0/4221902304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880b1a80 msgr2=0x7f6b880d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.072+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/4221902304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880b1a80 0x7f6b880d2a90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009dc0 tx=0x7f6b8000b330 comp rx=0 tx=0).stop 2023-07-06T21:41:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.073+0000 7f6b8cfff700 1 -- 172.21.15.84:0/4221902304 shutdown_connections 2023-07-06T21:41:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.073+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/4221902304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880b1a80 0x7f6b880d2a90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.073+0000 7f6b8cfff700 1 -- 172.21.15.84:0/4221902304 >> 172.21.15.84:0/4221902304 conn(0x7f6b8806b2b0 msgr2=0x7f6b8806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.073+0000 7f6b8cfff700 1 -- 172.21.15.84:0/4221902304 shutdown_connections 2023-07-06T21:41:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.073+0000 7f6b8cfff700 1 -- 172.21.15.84:0/4221902304 wait complete. 2023-07-06T21:41:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.074+0000 7f6b8cfff700 1 Processor -- start 2023-07-06T21:41:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.075+0000 7f6b8cfff700 1 -- start start 2023-07-06T21:41:26.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.075+0000 7f6b8cfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b880b1a80 0x7f6b88153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b8cfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 0x7f6b8815f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b8cfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880cfd10 0x7f6b880b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b8cfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b88159230 con 0x7f6b880b81c0 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b8cfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b881585b0 con 0x7f6b880b1a80 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b8cfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b88158890 con 0x7f6b880cfd10 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 0x7f6b8815f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 0x7f6b8815f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49530/0 (socket says 172.21.15.84:49530) 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 -- 172.21.15.84:0/908508327 learned_addr learned my addr 172.21.15.84:0/908508327 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b8659c700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b880b1a80 0x7f6b88153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b86d9d700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880cfd10 0x7f6b880b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 -- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880cfd10 msgr2=0x7f6b880b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880cfd10 0x7f6b880b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 -- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b880b1a80 msgr2=0x7f6b88153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b880b1a80 0x7f6b88153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 -- 172.21.15.84:0/908508327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80009a70 con 0x7f6b880b81c0 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b86d9d700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880cfd10 0x7f6b880b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.076+0000 7f6b7ffff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 0x7f6b8815f250 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c500 tx=0x7f6b7000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.077+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b700054e0 con 0x7f6b880b81c0 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.077+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70003680 con 0x7f6b880b81c0 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.077+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70003a30 con 0x7f6b880b81c0 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.077+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b88158b70 con 0x7f6b880b81c0 2023-07-06T21:41:26.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.077+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b88153960 con 0x7f6b880b81c0 2023-07-06T21:41:26.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.078+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b880666c0 con 0x7f6b880b81c0 2023-07-06T21:41:26.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.079+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6b70003b90 con 0x7f6b880b81c0 2023-07-06T21:41:26.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.079+0000 7f6b7effd700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b6806ace0 0x7f6b6806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.080+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6b70088b70 con 0x7f6b880b81c0 2023-07-06T21:41:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.080+0000 7f6b8659c700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b6806ace0 0x7f6b6806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.080+0000 7f6b8659c700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b6806ace0 0x7f6b6806d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b780011e0 tx=0x7f6b78001230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.084+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6b700539f0 con 0x7f6b880b81c0 2023-07-06T21:41:26.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.281+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b880cd2d0 con 0x7f6b880b81c0 2023-07-06T21:41:26.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.281+0000 7f6b7effd700 1 -- 172.21.15.84:0/908508327 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b700170a0 con 0x7f6b880b81c0 2023-07-06T21:41:26.285 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b6806ace0 msgr2=0x7f6b6806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b6806ace0 0x7f6b6806d190 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6b780011e0 tx=0x7f6b78001230 comp rx=0 tx=0).stop 2023-07-06T21:41:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 msgr2=0x7f6b8815f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 0x7f6b8815f250 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b7000c500 tx=0x7f6b7000c810 comp rx=0 tx=0).stop 2023-07-06T21:41:26.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 shutdown_connections 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6b6806ace0 0x7f6b6806d190 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b880cfd10 0x7f6b880b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b880b1a80 0x7f6b88153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 --2- 172.21.15.84:0/908508327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6b880b81c0 0x7f6b8815f250 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 >> 172.21.15.84:0/908508327 conn(0x7f6b8806b2b0 msgr2=0x7f6b880d2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 shutdown_connections 2023-07-06T21:41:26.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.284+0000 7f6b8cfff700 1 -- 172.21.15.84:0/908508327 wait complete. 2023-07-06T21:41:26.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:26.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.498+0000 7f3f66dff700 1 Processor -- start 2023-07-06T21:41:26.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.498+0000 7f3f66dff700 1 -- start start 2023-07-06T21:41:26.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.499+0000 7f3f66dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f600cfd10 0x7f3f600d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f66dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f600b1a80 0x7f3f600d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f66dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f600d1830 con 0x7f3f600b81c0 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f66dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f600cf330 con 0x7f3f600cfd10 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f66dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f600d1270 con 0x7f3f600b1a80 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f5ffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f600cfd10 0x7f3f600d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f64b9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f600b81c0 0x7f3f600cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43692/0 (socket says 172.21.15.84:43692) 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f64b9b700 1 -- 172.21.15.84:0/3029042608 learned_addr learned my addr 172.21.15.84:0/3029042608 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:26.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f6539c700 1 --2- 172.21.15.84:0/3029042608 >> v2:172.21.15.107:3300/0 conn(0x7f3f600b1a80 0x7f3f600d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f5f7fe700 1 -- 172.21.15.84:0/3029042608 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663197673 0 0) 0x7f3f600d1830 con 0x7f3f600b81c0 2023-07-06T21:41:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f5ffff700 1 -- 172.21.15.84:0/3029042608 >> v2:172.21.15.107:3300/0 conn(0x7f3f600b1a80 msgr2=0x7f3f600d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f5ffff700 1 --2- 172.21.15.84:0/3029042608 >> v2:172.21.15.107:3300/0 conn(0x7f3f600b1a80 0x7f3f600d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f5ffff700 1 -- 172.21.15.84:0/3029042608 >> v1:172.21.15.84:6789/0 conn(0x7f3f600b81c0 legacy=0x7f3f600cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.500+0000 7f3f5ffff700 1 -- 172.21.15.84:0/3029042608 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f600d1550 con 0x7f3f600cfd10 2023-07-06T21:41:26.505 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f5ffff700 1 --2- 172.21.15.84:0/3029042608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600cfd10 0x7f3f600d2380 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3f600b8d30 tx=0x7f3f4c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564a0625528c5673 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f5f7fe700 1 -- 172.21.15.84:0/3029042608 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c011830 con 0x7f3f600cfd10 2023-07-06T21:41:26.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f5f7fe700 1 -- 172.21.15.84:0/3029042608 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c011990 con 0x7f3f600cfd10 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f5f7fe700 1 -- 172.21.15.84:0/3029042608 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c011d20 con 0x7f3f600cfd10 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 -- 172.21.15.84:0/3029042608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600cfd10 msgr2=0x7f3f600d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 --2- 172.21.15.84:0/3029042608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600cfd10 0x7f3f600d2380 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3f600b8d30 tx=0x7f3f4c009b20 comp rx=0 tx=0).stop 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 -- 172.21.15.84:0/3029042608 shutdown_connections 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 --2- 172.21.15.84:0/3029042608 >> v2:172.21.15.107:3300/0 conn(0x7f3f600b1a80 0x7f3f600d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 --2- 172.21.15.84:0/3029042608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600cfd10 0x7f3f600d2380 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 -- 172.21.15.84:0/3029042608 >> 172.21.15.84:0/3029042608 conn(0x7f3f6006b2b0 msgr2=0x7f3f6006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:26.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.501+0000 7f3f66dff700 1 -- 172.21.15.84:0/3029042608 shutdown_connections 2023-07-06T21:41:26.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.502+0000 7f3f66dff700 1 -- 172.21.15.84:0/3029042608 wait complete. 2023-07-06T21:41:26.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.504+0000 7f3f66dff700 1 Processor -- start 2023-07-06T21:41:26.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.504+0000 7f3f66dff700 1 -- start start 2023-07-06T21:41:26.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f66dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 0x7f3f60155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f66dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f600b81c0 0x7f3f6015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.509 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f66dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f600cfd10 0x7f3f6015ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f66dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f60156b00 con 0x7f3f600b1a80 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f66dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6015dd60 con 0x7f3f600b81c0 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f66dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6015e040 con 0x7f3f600cfd10 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.505+0000 7f3f64b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 0x7f3f60155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 0x7f3f60155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49536/0 (socket says 172.21.15.84:49536) 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 -- 172.21.15.84:0/915351807 learned_addr learned my addr 172.21.15.84:0/915351807 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 -- 172.21.15.84:0/915351807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f600cfd10 msgr2=0x7f3f6015ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.510 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f600cfd10 0x7f3f6015ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 -- 172.21.15.84:0/915351807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f600b81c0 msgr2=0x7f3f6015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f600b81c0 0x7f3f6015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 -- 172.21.15.84:0/915351807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0094a0 con 0x7f3f600b1a80 2023-07-06T21:41:26.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f64b9b700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 0x7f3f60155d00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3f540011e0 tx=0x7f3f5400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54004c80 con 0x7f3f600b1a80 2023-07-06T21:41:26.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f54004e00 con 0x7f3f600b1a80 2023-07-06T21:41:26.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f54007310 con 0x7f3f600b1a80 2023-07-06T21:41:26.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.506+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6015e320 con 0x7f3f600b1a80 2023-07-06T21:41:26.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.507+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f60156240 con 0x7f3f600b1a80 2023-07-06T21:41:26.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.507+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f600b8900 con 0x7f3f600b1a80 2023-07-06T21:41:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.508+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f54007470 con 0x7f3f600b1a80 2023-07-06T21:41:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.510+0000 7f3f5d7fa700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4406ace0 0x7f3f4406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.510+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f3f540893c0 con 0x7f3f600b1a80 2023-07-06T21:41:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.510+0000 7f3f5ffff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4406ace0 0x7f3f4406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.510+0000 7f3f5ffff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4406ace0 0x7f3f4406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f600d0100 tx=0x7f3f4c010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.516 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.513+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f540542d0 con 0x7f3f600b1a80 2023-07-06T21:41:26.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.706+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f3f6004e1d0 con 0x7f3f600b1a80 2023-07-06T21:41:26.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.707+0000 7f3f5d7fa700 1 -- 172.21.15.84:0/915351807 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7f3f540578f0 con 0x7f3f600b1a80 2023-07-06T21:41:26.710 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4406ace0 msgr2=0x7f3f4406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4406ace0 0x7f3f4406d190 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3f600d0100 tx=0x7f3f4c010000 comp rx=0 tx=0).stop 2023-07-06T21:41:26.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 msgr2=0x7f3f60155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 0x7f3f60155d00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3f540011e0 tx=0x7f3f5400cae0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 shutdown_connections 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f4406ace0 0x7f3f4406d190 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f600cfd10 0x7f3f6015ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f600b1a80 0x7f3f60155d00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 --2- 172.21.15.84:0/915351807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f600b81c0 0x7f3f6015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 >> 172.21.15.84:0/915351807 conn(0x7f3f6006b2b0 msgr2=0x7f3f6015c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:26.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 shutdown_connections 2023-07-06T21:41:26.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.709+0000 7f3f66dff700 1 -- 172.21.15.84:0/915351807 wait complete. 2023-07-06T21:41:26.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:41:26.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.921+0000 7fb6761ff700 1 Processor -- start 2023-07-06T21:41:26.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.921+0000 7fb6761ff700 1 -- start start 2023-07-06T21:41:26.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb6761ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb6700c9410 0x7fb67015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb6761ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb670070230 con 0x7fb67015a7e0 2023-07-06T21:41:26.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb6761ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67015b2a0 con 0x7fb670154910 2023-07-06T21:41:26.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb6761ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb670071390 con 0x7fb6700c9410 2023-07-06T21:41:26.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb66effd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb67015a7e0 0x7fb6700c8d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:43698/0 (socket says 172.21.15.84:43698) 2023-07-06T21:41:26.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb66effd700 1 -- 172.21.15.84:0/4024575142 learned_addr learned my addr 172.21.15.84:0/4024575142 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb66ffff700 1 --2- 172.21.15.84:0/4024575142 >> v2:172.21.15.107:3300/0 conn(0x7fb6700c9410 0x7fb67015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb66e7fc700 1 -- 172.21.15.84:0/4024575142 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453359705 0 0) 0x7fb670070230 con 0x7fb67015a7e0 2023-07-06T21:41:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.923+0000 7fb66e7fc700 1 -- 172.21.15.84:0/4024575142 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6580035b0 con 0x7fb67015a7e0 2023-07-06T21:41:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66ffff700 1 -- 172.21.15.84:0/4024575142 >> v1:172.21.15.107:6789/0 conn(0x7fb670154910 legacy=0x7fb670070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66ffff700 1 -- 172.21.15.84:0/4024575142 >> v1:172.21.15.84:6789/0 conn(0x7fb67015a7e0 legacy=0x7fb6700c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66e7fc700 1 -- 172.21.15.84:0/4024575142 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776595142 0 0) 0x7fb6580035b0 con 0x7fb67015a7e0 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66ffff700 1 -- 172.21.15.84:0/4024575142 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67015b580 con 0x7fb6700c9410 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66ffff700 1 --2- 172.21.15.84:0/4024575142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015ac30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb668009d90 tx=0x7fb66800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39ee5403b3e313e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66e7fc700 1 -- 172.21.15.84:0/4024575142 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66800e070 con 0x7fb6700c9410 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66e7fc700 1 -- 172.21.15.84:0/4024575142 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb668003a20 con 0x7fb6700c9410 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.924+0000 7fb66e7fc700 1 -- 172.21.15.84:0/4024575142 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664001670 con 0x7fb6700c9410 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 -- 172.21.15.84:0/4024575142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 msgr2=0x7fb67015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 --2- 172.21.15.84:0/4024575142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015ac30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb668009d90 tx=0x7fb66800b330 comp rx=0 tx=0).stop 2023-07-06T21:41:26.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 -- 172.21.15.84:0/4024575142 shutdown_connections 2023-07-06T21:41:26.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 --2- 172.21.15.84:0/4024575142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015ac30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 -- 172.21.15.84:0/4024575142 >> 172.21.15.84:0/4024575142 conn(0x7fb6700cf3d0 msgr2=0x7fb6700cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:26.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 -- 172.21.15.84:0/4024575142 shutdown_connections 2023-07-06T21:41:26.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.925+0000 7fb6761ff700 1 -- 172.21.15.84:0/4024575142 wait complete. 2023-07-06T21:41:26.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.927+0000 7fb6761ff700 1 Processor -- start 2023-07-06T21:41:26.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.927+0000 7fb6761ff700 1 -- start start 2023-07-06T21:41:26.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb6761ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb6761ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 0x7fb670150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb6761ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb67015a7e0 0x7fb670159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb6761ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700cc700 con 0x7fb670154910 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb6761ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67015d950 con 0x7fb6700c9410 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb6761ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67015dc30 con 0x7fb67015a7e0 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 0x7fb670150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 0x7fb670150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49538/0 (socket says 172.21.15.84:49538) 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 -- 172.21.15.84:0/2915898317 learned_addr learned my addr 172.21.15.84:0/2915898317 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:26.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66f7fe700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66ffff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb67015a7e0 0x7fb670159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 -- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb67015a7e0 msgr2=0x7fb670159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb67015a7e0 0x7fb670159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 -- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 msgr2=0x7fb67015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66effd700 1 -- 172.21.15.84:0/2915898317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668009a70 con 0x7fb670154910 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66ffff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb67015a7e0 0x7fb670159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:26.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.928+0000 7fb66f7fe700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.929+0000 7fb66effd700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 0x7fb670150870 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c750 tx=0x7fb66400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.929+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664004c80 con 0x7fb670154910 2023-07-06T21:41:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.929+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb664004e00 con 0x7fb670154910 2023-07-06T21:41:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.929+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb664007310 con 0x7fb670154910 2023-07-06T21:41:26.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.929+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67015df10 con 0x7fb670154910 2023-07-06T21:41:26.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.929+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6701505b0 con 0x7fb670154910 2023-07-06T21:41:26.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.930+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6700cc840 con 0x7fb670154910 2023-07-06T21:41:26.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.931+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb664007470 con 0x7fb670154910 2023-07-06T21:41:26.936 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.932+0000 7fb657fff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb65c06ace0 0x7fb65c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:26.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.932+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb664089370 con 0x7fb670154910 2023-07-06T21:41:26.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.932+0000 7fb66f7fe700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb65c06ace0 0x7fb65c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:26.937 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.933+0000 7fb66f7fe700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb65c06ace0 0x7fb65c06d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb660000f60 tx=0x7fb660009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:26.939 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:26.936+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb6700cc840 con 0x7fb670154910 2023-07-06T21:41:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.151+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb6700cc840 con 0x7fb670154910 2023-07-06T21:41:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.151+0000 7fb657fff700 1 -- 172.21.15.84:0/2915898317 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb6700cc840 con 0x7fb670154910 2023-07-06T21:41:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb65c06ace0 msgr2=0x7fb65c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb65c06ace0 0x7fb65c06d190 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb660000f60 tx=0x7fb660009040 comp rx=0 tx=0).stop 2023-07-06T21:41:27.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 msgr2=0x7fb670150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 0x7fb670150870 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb66400c750 tx=0x7fb66400cb10 comp rx=0 tx=0).stop 2023-07-06T21:41:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 shutdown_connections 2023-07-06T21:41:27.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb65c06ace0 0x7fb65c06d190 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb67015a7e0 0x7fb670159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6700c9410 0x7fb67015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 --2- 172.21.15.84:0/2915898317 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb670154910 0x7fb670150870 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 >> 172.21.15.84:0/2915898317 conn(0x7fb6700cf3d0 msgr2=0x7fb6700cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 shutdown_connections 2023-07-06T21:41:27.161 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:27.154+0000 7fb6761ff700 1 -- 172.21.15.84:0/2915898317 wait complete. 2023-07-06T21:41:27.162 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:41:27.175 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:41:27.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:27.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:27.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[148692]: fuse finished with error 0 and tester_r 0 2023-07-06T21:41:27.247 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:41:33.349 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:41:33.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:33.350 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:41:33.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2023-07-06T21:41:33.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.599+0000 7f47489ff700 1 Processor -- start 2023-07-06T21:41:33.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.599+0000 7f47489ff700 1 -- start start 2023-07-06T21:41:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.600+0000 7f47489ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440d0ea0 0x7f47440d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.600+0000 7f47489ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4744160ac0 0x7f47440cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.600+0000 7f47489ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f47440cfe20 0x7f474415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.600+0000 7f47489ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47440ceef0 con 0x7f4744160ac0 2023-07-06T21:41:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.600+0000 7f47489ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47440cf450 con 0x7f47440cfe20 2023-07-06T21:41:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.600+0000 7f47489ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47440cf120 con 0x7f47440d0ea0 2023-07-06T21:41:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4744160ac0 0x7f47440cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4744160ac0 0x7f47440cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34016/0 (socket says 172.21.15.84:34016) 2023-07-06T21:41:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 -- 172.21.15.84:0/3237738204 learned_addr learned my addr 172.21.15.84:0/3237738204 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f474259c700 1 --2- 172.21.15.84:0/3237738204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440d0ea0 0x7f47440d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f4742d9d700 1 --2- 172.21.15.84:0/3237738204 >> v2:172.21.15.107:3300/0 conn(0x7f47440cfe20 0x7f474415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 -- 172.21.15.84:0/3237738204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440d0ea0 msgr2=0x7f47440d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 --2- 172.21.15.84:0/3237738204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440d0ea0 0x7f47440d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 -- 172.21.15.84:0/3237738204 >> v2:172.21.15.107:3300/0 conn(0x7f47440cfe20 msgr2=0x7f474415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 --2- 172.21.15.84:0/3237738204 >> v2:172.21.15.107:3300/0 conn(0x7f47440cfe20 0x7f474415fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 -- 172.21.15.84:0/3237738204 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47440cd420 con 0x7f4744160ac0 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.601+0000 7f473bfff700 1 --2- 172.21.15.84:0/3237738204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440cf8e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47440b49f0 tx=0x7f472c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219899e64cddd43e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f4741d9b700 1 -- 172.21.15.84:0/3237738204 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00b870 con 0x7f4744160ac0 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f4741d9b700 1 -- 172.21.15.84:0/3237738204 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c00b9d0 con 0x7f4744160ac0 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f4741d9b700 1 -- 172.21.15.84:0/3237738204 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00bd60 con 0x7f4744160ac0 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 -- 172.21.15.84:0/3237738204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 msgr2=0x7f47440cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 --2- 172.21.15.84:0/3237738204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440cf8e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47440b49f0 tx=0x7f472c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 -- 172.21.15.84:0/3237738204 shutdown_connections 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 --2- 172.21.15.84:0/3237738204 >> v2:172.21.15.107:3300/0 conn(0x7f47440cfe20 0x7f474415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 --2- 172.21.15.84:0/3237738204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440d0ea0 0x7f47440d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 --2- 172.21.15.84:0/3237738204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440cf8e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 -- 172.21.15.84:0/3237738204 >> 172.21.15.84:0/3237738204 conn(0x7f47440715c0 msgr2=0x7f4744071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 -- 172.21.15.84:0/3237738204 shutdown_connections 2023-07-06T21:41:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.602+0000 7f47489ff700 1 -- 172.21.15.84:0/3237738204 wait complete. 2023-07-06T21:41:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.604+0000 7f47489ff700 1 Processor -- start 2023-07-06T21:41:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.604+0000 7f47489ff700 1 -- start start 2023-07-06T21:41:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f47489ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440cfe20 0x7f47440c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f47489ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f47489ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47440ca2b0 0x7f47440b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f47489ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47440ca750 con 0x7f4744160ac0 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f47489ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47440c58c0 con 0x7f47440ca2b0 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f47489ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47440c7860 con 0x7f47440cfe20 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.605+0000 7f473bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f474259c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440cfe20 0x7f47440c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34018/0 (socket says 172.21.15.84:34018) 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 -- 172.21.15.84:0/2658828949 learned_addr learned my addr 172.21.15.84:0/2658828949 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f4742d9d700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47440ca2b0 0x7f47440b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 -- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440cfe20 msgr2=0x7f47440c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440cfe20 0x7f47440c5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 -- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47440ca2b0 msgr2=0x7f47440b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47440ca2b0 0x7f47440b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 -- 172.21.15.84:0/2658828949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c009550 con 0x7f4744160ac0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473bfff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440c9d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f472c009c10 tx=0x7f472c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c004480 con 0x7f4744160ac0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c0045e0 con 0x7f4744160ac0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47440b8540 con 0x7f4744160ac0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.606+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c0038e0 con 0x7f4744160ac0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.607+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47440c7210 con 0x7f4744160ac0 2023-07-06T21:41:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.608+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474404e1d0 con 0x7f4744160ac0 2023-07-06T21:41:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.608+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f472c021070 con 0x7f4744160ac0 2023-07-06T21:41:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.609+0000 7f473affd700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f472406ace0 0x7f472406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.609+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f472c08aab0 con 0x7f4744160ac0 2023-07-06T21:41:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.610+0000 7f474259c700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f472406ace0 0x7f472406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.610+0000 7f474259c700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f472406ace0 0x7f472406d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4734006fd0 tx=0x7f4734008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.614+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f472c0559c0 con 0x7f4744160ac0 2023-07-06T21:41:33.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.895+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) v1 -- 0x7f47440c79a0 con 0x7f4744160ac0 2023-07-06T21:41:33.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.899+0000 7f473affd700 1 -- 172.21.15.84:0/2658828949 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v12) v1 ==== 211+0+0 (secure 0 0 0) 0x7f472c015100 con 0x7f4744160ac0 2023-07-06T21:41:33.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f472406ace0 msgr2=0x7f472406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:33.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f472406ace0 0x7f472406d190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4734006fd0 tx=0x7f4734008040 comp rx=0 tx=0).stop 2023-07-06T21:41:33.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 msgr2=0x7f47440c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:33.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440c9d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f472c009c10 tx=0x7f472c004750 comp rx=0 tx=0).stop 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 shutdown_connections 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f472406ace0 0x7f472406d190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47440ca2b0 0x7f47440b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47440cfe20 0x7f47440c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 --2- 172.21.15.84:0/2658828949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4744160ac0 0x7f47440c9d70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 >> 172.21.15.84:0/2658828949 conn(0x7f47440715c0 msgr2=0x7f47440d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 shutdown_connections 2023-07-06T21:41:33.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:33.902+0000 7f47489ff700 1 -- 172.21.15.84:0/2658828949 wait complete. 2023-07-06T21:41:33.907 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:41:33.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:41:34.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.116+0000 7fc85e1ff700 1 Processor -- start 2023-07-06T21:41:34.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.116+0000 7fc85e1ff700 1 -- start start 2023-07-06T21:41:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.117+0000 7fc85e1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc858158830 0x7fc8580c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.117+0000 7fc85e1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580c3f80 con 0x7fc8580c5280 2023-07-06T21:41:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.117+0000 7fc85e1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8580c4260 con 0x7fc85806a5a0 2023-07-06T21:41:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.117+0000 7fc85e1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8580c4540 con 0x7fc858158830 2023-07-06T21:41:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.117+0000 7fc856ffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc8580c5280 0x7fc858069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42764/0 (socket says 172.21.15.84:42764) 2023-07-06T21:41:34.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.117+0000 7fc856ffd700 1 -- 172.21.15.84:0/3575044106 learned_addr learned my addr 172.21.15.84:0/3575044106 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:34.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807630683 0 0) 0x7fc8580c3f80 con 0x7fc8580c5280 2023-07-06T21:41:34.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8577fe700 1 --2- 172.21.15.84:0/3575044106 >> v2:172.21.15.107:3300/0 conn(0x7fc858158830 0x7fc8580c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8400035b0 con 0x7fc8580c5280 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2198737096 0 0) 0x7fc8400035b0 con 0x7fc8580c5280 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 >> v2:172.21.15.107:3300/0 conn(0x7fc858158830 msgr2=0x7fc8580c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 --2- 172.21.15.84:0/3575044106 >> v2:172.21.15.107:3300/0 conn(0x7fc858158830 0x7fc8580c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 >> v1:172.21.15.107:6789/0 conn(0x7fc85806a5a0 legacy=0x7fc8580c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580c4820 con 0x7fc8580c5280 2023-07-06T21:41:34.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fc84c001670 con 0x7fc8580c5280 2023-07-06T21:41:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 >> v1:172.21.15.84:6789/0 conn(0x7fc8580c5280 legacy=0x7fc858069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc840003a50 0x7fc840003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc840004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc840005320 0x7fc84000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580c4540 con 0x7fc840004850 2023-07-06T21:41:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84000c210 con 0x7fc840005320 2023-07-06T21:41:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.118+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84000c3d0 con 0x7fc840003a50 2023-07-06T21:41:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc840004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc857fff700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc840005320 0x7fc84000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc8577fe700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc840003a50 0x7fc840003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 -- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc840003a50 msgr2=0x7fc840003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc840003a50 0x7fc840003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 -- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc840005320 msgr2=0x7fc84000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc840005320 0x7fc84000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 -- 172.21.15.84:0/3575044106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400035b0 con 0x7fc840004850 2023-07-06T21:41:34.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc857fff700 1 -- 172.21.15.84:0/3575044106 reap_dead start 2023-07-06T21:41:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc856ffd700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc840004cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00c160 tx=0x7fc840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d50e81140592c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c007b70 con 0x7fc840004850 2023-07-06T21:41:34.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84c00cd00 con 0x7fc840004850 2023-07-06T21:41:34.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.119+0000 7fc8567fc700 1 -- 172.21.15.84:0/3575044106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84c005ba0 con 0x7fc840004850 2023-07-06T21:41:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 -- 172.21.15.84:0/3575044106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 msgr2=0x7fc840004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc840004cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc84c00c160 tx=0x7fc840005880 comp rx=0 tx=0).stop 2023-07-06T21:41:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 -- 172.21.15.84:0/3575044106 shutdown_connections 2023-07-06T21:41:34.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/3575044106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc840004cc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 -- 172.21.15.84:0/3575044106 >> 172.21.15.84:0/3575044106 conn(0x7fc8580cf3d0 msgr2=0x7fc8580cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 -- 172.21.15.84:0/3575044106 shutdown_connections 2023-07-06T21:41:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.120+0000 7fc85e1ff700 1 -- 172.21.15.84:0/3575044106 wait complete. 2023-07-06T21:41:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.122+0000 7fc85e1ff700 1 Processor -- start 2023-07-06T21:41:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.122+0000 7fc85e1ff700 1 -- start start 2023-07-06T21:41:34.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc85e1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc8580aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc85e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8580c7950 0x7fc8580c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc85e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8580af330 0x7fc8580af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc85e1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580c8300 con 0x7fc840004850 2023-07-06T21:41:34.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc85e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580afce0 con 0x7fc8580af330 2023-07-06T21:41:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc85e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc858158f20 con 0x7fc8580c7950 2023-07-06T21:41:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc8580aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc8577fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc8580aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34034/0 (socket says 172.21.15.84:34034) 2023-07-06T21:41:34.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc8577fe700 1 -- 172.21.15.84:0/518627383 learned_addr learned my addr 172.21.15.84:0/518627383 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc857fff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8580af330 0x7fc8580af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.123+0000 7fc856ffd700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8580c7950 0x7fc8580c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc8577fe700 1 -- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8580c7950 msgr2=0x7fc8580c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc8577fe700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8580c7950 0x7fc8580c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc8577fe700 1 -- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8580af330 msgr2=0x7fc8580af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc8577fe700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8580af330 0x7fc8580af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc8577fe700 1 -- 172.21.15.84:0/518627383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858159200 con 0x7fc840004850 2023-07-06T21:41:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc8577fe700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc8580aeac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc8480087f0 tx=0x7fc848008b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:34.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848009cd0 con 0x7fc840004850 2023-07-06T21:41:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc848009e30 con 0x7fc840004850 2023-07-06T21:41:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8581594e0 con 0x7fc840004850 2023-07-06T21:41:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848016620 con 0x7fc840004850 2023-07-06T21:41:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.124+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858159aa0 con 0x7fc840004850 2023-07-06T21:41:34.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.125+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc858159c60 con 0x7fc840004850 2023-07-06T21:41:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.126+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc848016780 con 0x7fc840004850 2023-07-06T21:41:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.127+0000 7fc83ffff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc84406ace0 0x7fc84406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:34.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.127+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc848089430 con 0x7fc840004850 2023-07-06T21:41:34.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.127+0000 7fc856ffd700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc84406ace0 0x7fc84406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:34.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.128+0000 7fc856ffd700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc84406ace0 0x7fc84406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:34.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.132+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc848054360 con 0x7fc840004850 2023-07-06T21:41:34.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.375+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8580c39b0 con 0x7fc840004850 2023-07-06T21:41:34.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.375+0000 7fc83ffff700 1 -- 172.21.15.84:0/518627383 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+82 (secure 0 0 0) 0x7fc84801b070 con 0x7fc840004850 2023-07-06T21:41:34.378 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2023-07-06T21:41:34.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc84406ace0 msgr2=0x7fc84406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc84406ace0 0x7fc84406d190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc84c000c00 tx=0x7fc84c007000 comp rx=0 tx=0).stop 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 msgr2=0x7fc8580aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc8580aeac0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc8480087f0 tx=0x7fc848008b00 comp rx=0 tx=0).stop 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 shutdown_connections 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc84406ace0 0x7fc84406d190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8580af330 0x7fc8580af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc840004850 0x7fc8580aeac0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 --2- 172.21.15.84:0/518627383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8580c7950 0x7fc8580c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:34.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 >> 172.21.15.84:0/518627383 conn(0x7fc8580cf3d0 msgr2=0x7fc8580ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:34.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.377+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 shutdown_connections 2023-07-06T21:41:34.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:34.378+0000 7fc85e1ff700 1 -- 172.21.15.84:0/518627383 wait complete. 2023-07-06T21:41:34.399 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:41:34.399 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:41:34.399 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:41:34.400 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:41:34.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:34.400 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:41:34.400 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:41:34.401 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:41:34.401 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:34.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:34.402 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:41:34.445 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:41:34.445 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:41:34.446 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:41:34.446 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.446 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:41:34.447 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.447 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:34.447 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:34.447 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:41:34.448 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:34.448 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:34.448 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:41:34.448 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:41:34.448 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:34.449 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:41:34.449 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:34.449 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:41:34.449 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42108sec preferred_lft 42108sec 2023-07-06T21:41:34.449 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.450 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:41:34.451 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:41:34.451 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:41:34.451 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.451 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:41:34.451 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:41:34.451 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4492:bbff:fe16:eebb/64 scope link 2023-07-06T21:41:34.452 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:41:34.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:34.452 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:41:34.550 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:41:34.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:34.553 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:41:34.600 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:41:34.600 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:41:34.600 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:41:34.601 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:41:34.601 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:41:34.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:34.602 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:41:34.602 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:41:34.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:41:34.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:41:34.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:41:34.604 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:41:34.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:34.810 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:41:34.853 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:41:34.854 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:41:34.854 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:34.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:34.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:41:34.961 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:41:34.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:41:34.962 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:35.004 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:41:35.004 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:41:35.005 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:35.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:41:35.193 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:41:35.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:41:35.259 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:41:35.261 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:41:35.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:35.262 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:41:35.305 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:41:35.306 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:41:35.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:41:35.420 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:41:35.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:41:35.487 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:41:35.490 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:41:35.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:35.491 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:41:35.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:41:35.514+0000 7f5b849fb5c0 -1 init, newargv = 0x55ac0a7c2de0 newargc=15 2023-07-06T21:41:35.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[149998]: starting ceph client 2023-07-06T21:41:35.526 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[149998]: starting fuse 2023-07-06T21:41:35.537 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:41:35.537 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:41:35.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:41:35.538 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:41:35.538 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:41:35.538 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:41:35.539 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:41:35.539 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:35.539 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:41:35.540 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:41:35.540 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:41:35.540 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:41:35.540 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:35.541 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:41:35.541 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:41:35.541 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:41:35.542 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:41:35.542 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:35.542 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:41:35.542 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:41:35.543 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:41:35.543 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:41:35.543 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:41:35.544 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:41:35.544 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:41:35.544 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:41:35.544 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:41:35.545 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:41:35.545 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:41:35.718 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.149998.asok 2023-07-06T21:41:35.896 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.149998.asok 2023-07-06T21:41:35.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:35.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.149998.asok status 2023-07-06T21:41:36.118 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:41:36.119 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "149998", 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 5757, 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:41:36.120 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 5757 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 786380903 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:41:36.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 786380903 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.5757 192.168.0.1:0/786380903", 2023-07-06T21:41:36.122 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/786380903", 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 16, 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 40, 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:41:36.123 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:41:36.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:36.133 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:41:36.133 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:36.177 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:41:36.177 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:36.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:36.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:36.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:36.284 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2023-07-06T21:41:36.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:36.330 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2023-07-06T21:41:36.418 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:36.419 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:36.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:36.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.692+0000 7f9f173ff700 1 Processor -- start 2023-07-06T21:41:36.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.692+0000 7f9f173ff700 1 -- start start 2023-07-06T21:41:36.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f173ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f100b1260 0x7f9f100cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:36.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f173ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f1006ba90 0x7f9f100b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:36.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f173ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f10160960 con 0x7f9f1006ee90 2023-07-06T21:41:36.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f173ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1006a5b0 con 0x7f9f100b1260 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f173ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f10160e90 con 0x7f9f1006ba90 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f1519b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f100b1260 0x7f9f100cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.693+0000 7f9f1599c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9f1006ee90 0x7f9f1006f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42778/0 (socket says 172.21.15.84:42778) 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1599c700 1 -- 172.21.15.84:0/2982304350 learned_addr learned my addr 172.21.15.84:0/2982304350 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1499a700 1 --2- 172.21.15.84:0/2982304350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f1006ba90 0x7f9f100b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1519b700 1 -- 172.21.15.84:0/2982304350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f1006ba90 msgr2=0x7f9f100b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:36.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1519b700 1 --2- 172.21.15.84:0/2982304350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f1006ba90 0x7f9f100b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1519b700 1 -- 172.21.15.84:0/2982304350 >> v1:172.21.15.84:6789/0 conn(0x7f9f1006ee90 legacy=0x7f9f1006f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:36.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1519b700 1 -- 172.21.15.84:0/2982304350 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1015c820 con 0x7f9f100b1260 2023-07-06T21:41:36.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f07fff700 1 -- 172.21.15.84:0/2982304350 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618761367 0 0) 0x7f9f10160960 con 0x7f9f1006ee90 2023-07-06T21:41:36.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f1519b700 1 --2- 172.21.15.84:0/2982304350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f100b1260 0x7f9f100cef60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009870 tx=0x7f9f0c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396a159ce1ba5493 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:36.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f07fff700 1 -- 172.21.15.84:0/2982304350 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c011830 con 0x7f9f100b1260 2023-07-06T21:41:36.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f07fff700 1 -- 172.21.15.84:0/2982304350 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c011990 con 0x7f9f100b1260 2023-07-06T21:41:36.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.694+0000 7f9f07fff700 1 -- 172.21.15.84:0/2982304350 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c011d20 con 0x7f9f100b1260 2023-07-06T21:41:36.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 -- 172.21.15.84:0/2982304350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f100b1260 msgr2=0x7f9f100cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:36.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 --2- 172.21.15.84:0/2982304350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f100b1260 0x7f9f100cef60 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009870 tx=0x7f9f0c009b80 comp rx=0 tx=0).stop 2023-07-06T21:41:36.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 -- 172.21.15.84:0/2982304350 shutdown_connections 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 --2- 172.21.15.84:0/2982304350 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f100b1260 0x7f9f100cef60 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 --2- 172.21.15.84:0/2982304350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f1006ba90 0x7f9f100b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 -- 172.21.15.84:0/2982304350 >> 172.21.15.84:0/2982304350 conn(0x7f9f100cc260 msgr2=0x7f9f100cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 -- 172.21.15.84:0/2982304350 shutdown_connections 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.695+0000 7f9f173ff700 1 -- 172.21.15.84:0/2982304350 wait complete. 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.697+0000 7f9f173ff700 1 Processor -- start 2023-07-06T21:41:36.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.697+0000 7f9f173ff700 1 -- start start 2023-07-06T21:41:36.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f173ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 0x7f9f101f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:36.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f173ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f1006ee90 0x7f9f1015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f173ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f100b1260 0x7f9f101e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f173ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f100c3700 con 0x7f9f1006ba90 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f173ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1015ee90 con 0x7f9f1006ee90 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f173ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1015f170 con 0x7f9f100b1260 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 0x7f9f101f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 0x7f9f101f1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34050/0 (socket says 172.21.15.84:34050) 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 -- 172.21.15.84:0/3925232140 learned_addr learned my addr 172.21.15.84:0/3925232140 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:36.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 -- 172.21.15.84:0/3925232140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f100b1260 msgr2=0x7f9f101e7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:36.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f100b1260 0x7f9f101e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 -- 172.21.15.84:0/3925232140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f1006ee90 msgr2=0x7f9f1015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:36.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f1006ee90 0x7f9f1015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.698+0000 7f9f1519b700 1 -- 172.21.15.84:0/3925232140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0094a0 con 0x7f9f1006ba90 2023-07-06T21:41:36.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.699+0000 7f9f1519b700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 0x7f9f101f1520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0097c0 tx=0x7f9f0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.699+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c0036a0 con 0x7f9f1006ba90 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.699+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c01f070 con 0x7f9f1006ba90 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.699+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f100c3390 con 0x7f9f1006ba90 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.699+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c003a60 con 0x7f9f1006ba90 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.699+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1015ea30 con 0x7f9f1006ba90 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.700+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1015ec20 con 0x7f9f1006ba90 2023-07-06T21:41:36.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.701+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f0c003bc0 con 0x7f9f1006ba90 2023-07-06T21:41:36.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.702+0000 7f9f05ffb700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ef406ace0 0x7f9ef406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:36.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.703+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9f0c021070 con 0x7f9f1006ba90 2023-07-06T21:41:36.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.703+0000 7f9f1499a700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ef406ace0 0x7f9ef406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:36.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.703+0000 7f9f1499a700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ef406ace0 0x7f9ef406d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9f1015f770 tx=0x7f9f00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:36.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.707+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f0c055dc0 con 0x7f9f1006ba90 2023-07-06T21:41:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.905+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f10071c50 con 0x7f9f1006ba90 2023-07-06T21:41:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.907+0000 7f9f05ffb700 1 -- 172.21.15.84:0/3925232140 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7f9f0c015090 con 0x7f9f1006ba90 2023-07-06T21:41:36.910 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:36.910 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ef406ace0 msgr2=0x7f9ef406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ef406ace0 0x7f9ef406d190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9f1015f770 tx=0x7f9f00008040 comp rx=0 tx=0).stop 2023-07-06T21:41:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 msgr2=0x7f9f101f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 0x7f9f101f1520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c0097c0 tx=0x7f9f0c004750 comp rx=0 tx=0).stop 2023-07-06T21:41:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 shutdown_connections 2023-07-06T21:41:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ef406ace0 0x7f9ef406d190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f100b1260 0x7f9f101e7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f1006ba90 0x7f9f101f1520 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 --2- 172.21.15.84:0/3925232140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f1006ee90 0x7f9f1015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:36.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 >> 172.21.15.84:0/3925232140 conn(0x7f9f100cc260 msgr2=0x7f9f100b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:36.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 shutdown_connections 2023-07-06T21:41:36.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:36.909+0000 7f9f173ff700 1 -- 172.21.15.84:0/3925232140 wait complete. 2023-07-06T21:41:36.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:36.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:37.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.127+0000 7f00d53ff700 1 Processor -- start 2023-07-06T21:41:37.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.128+0000 7f00d53ff700 1 -- start start 2023-07-06T21:41:37.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00d53ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d00ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00d53ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d00cf600 con 0x7f00d00cc3d0 2023-07-06T21:41:37.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00d53ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d00ccf40 con 0x7f00d0070ab0 2023-07-06T21:41:37.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00d53ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00c9b00 con 0x7f00d0159220 2023-07-06T21:41:37.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00c7fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f00d00cc3d0 0x7f00d00703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42794/0 (socket says 172.21.15.84:42794) 2023-07-06T21:41:37.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00c7fff700 1 -- 172.21.15.84:0/502550389 learned_addr learned my addr 172.21.15.84:0/502550389 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:37.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.129+0000 7f00ceffd700 1 --2- 172.21.15.84:0/502550389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d00ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640488835 0 0) 0x7f00d00cf600 con 0x7f00d00cc3d0 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00bc0035b0 con 0x7f00d00cc3d0 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2939501301 0 0) 0x7f00d00ccf40 con 0x7f00d0070ab0 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d00cf600 con 0x7f00d0070ab0 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ceffd700 1 -- 172.21.15.84:0/502550389 >> v1:172.21.15.107:6789/0 conn(0x7f00d0070ab0 legacy=0x7f00d006c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ceffd700 1 -- 172.21.15.84:0/502550389 >> v1:172.21.15.84:6789/0 conn(0x7f00d00cc3d0 legacy=0x7f00d00703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2149677920 0 0) 0x7f00bc0035b0 con 0x7f00d00cc3d0 2023-07-06T21:41:37.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ceffd700 1 -- 172.21.15.84:0/502550389 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d00c9de0 con 0x7f00d0159220 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.130+0000 7f00ceffd700 1 --2- 172.21.15.84:0/502550389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d00ca8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009870 tx=0x7f00c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b06a392f5a04156 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8001670 con 0x7f00d0159220 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c0004030 con 0x7f00d0159220 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00ce7fc700 1 -- 172.21.15.84:0/502550389 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00043c0 con 0x7f00d0159220 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 -- 172.21.15.84:0/502550389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 msgr2=0x7f00d00ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 --2- 172.21.15.84:0/502550389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d00ca8c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009870 tx=0x7f00c0009b80 comp rx=0 tx=0).stop 2023-07-06T21:41:37.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 -- 172.21.15.84:0/502550389 shutdown_connections 2023-07-06T21:41:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 --2- 172.21.15.84:0/502550389 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d00ca8c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 -- 172.21.15.84:0/502550389 >> 172.21.15.84:0/502550389 conn(0x7f00d00cbf10 msgr2=0x7f00d00ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 -- 172.21.15.84:0/502550389 shutdown_connections 2023-07-06T21:41:37.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.131+0000 7f00d53ff700 1 -- 172.21.15.84:0/502550389 wait complete. 2023-07-06T21:41:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.133+0000 7f00d53ff700 1 Processor -- start 2023-07-06T21:41:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.133+0000 7f00d53ff700 1 -- start start 2023-07-06T21:41:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00d53ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 0x7f00d01f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00d53ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d00cc3d0 0x7f00d00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00d53ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d0150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.139 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00d53ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00af640 con 0x7f00d0070ab0 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00d53ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d014a590 con 0x7f00d00cc3d0 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00d53ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00b0430 con 0x7f00d0159220 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00ceffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 0x7f00d01f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00ceffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 0x7f00d01f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34060/0 (socket says 172.21.15.84:34060) 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.134+0000 7f00ceffd700 1 -- 172.21.15.84:0/2337264595 learned_addr learned my addr 172.21.15.84:0/2337264595 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00c7fff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d00cc3d0 0x7f00d00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00cf7fe700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d0150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00ceffd700 1 -- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 msgr2=0x7f00d0150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00ceffd700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d0150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00ceffd700 1 -- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d00cc3d0 msgr2=0x7f00d00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00ceffd700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d00cc3d0 0x7f00d00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00ceffd700 1 -- 172.21.15.84:0/2337264595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c00094a0 con 0x7f00d0070ab0 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00c7fff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d00cc3d0 0x7f00d00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:37.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00ceffd700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 0x7f00d01f21c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0003e30 con 0x7f00d0070ab0 2023-07-06T21:41:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c001f070 con 0x7f00d0070ab0 2023-07-06T21:41:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d01f2700 con 0x7f00d0070ab0 2023-07-06T21:41:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.135+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0011870 con 0x7f00d0070ab0 2023-07-06T21:41:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.136+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d00afb00 con 0x7f00d0070ab0 2023-07-06T21:41:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.137+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d00cc800 con 0x7f00d0070ab0 2023-07-06T21:41:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.137+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f00c0021030 con 0x7f00d0070ab0 2023-07-06T21:41:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.138+0000 7f00c77fe700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00b006ace0 0x7f00b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.138+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f00c008aa00 con 0x7f00d0070ab0 2023-07-06T21:41:37.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.138+0000 7f00c7fff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00b006ace0 0x7f00b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.144 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.139+0000 7f00c7fff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00b006ace0 0x7f00b006d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00d01f5d30 tx=0x7f00b8001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.143+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f00c0055a30 con 0x7f00d0070ab0 2023-07-06T21:41:37.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.339+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00d004e1d0 con 0x7f00d0070ab0 2023-07-06T21:41:37.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.340+0000 7f00c77fe700 1 -- 172.21.15.84:0/2337264595 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00c0059050 con 0x7f00d0070ab0 2023-07-06T21:41:37.343 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:37.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00b006ace0 msgr2=0x7f00b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00b006ace0 0x7f00b006d190 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f00d01f5d30 tx=0x7f00b8001630 comp rx=0 tx=0).stop 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 msgr2=0x7f00d01f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 0x7f00d01f21c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f00c0000c00 tx=0x7f00c0004750 comp rx=0 tx=0).stop 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 shutdown_connections 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f00b006ace0 0x7f00b006d190 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00d0159220 0x7f00d0150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f00d0070ab0 0x7f00d01f21c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 --2- 172.21.15.84:0/2337264595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00d00cc3d0 0x7f00d00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 >> 172.21.15.84:0/2337264595 conn(0x7f00d00cbf10 msgr2=0x7f00d006d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:37.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 shutdown_connections 2023-07-06T21:41:37.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.342+0000 7f00d53ff700 1 -- 172.21.15.84:0/2337264595 wait complete. 2023-07-06T21:41:37.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:37.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.557+0000 7fd931bff700 1 Processor -- start 2023-07-06T21:41:37.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.558+0000 7fd931bff700 1 -- start start 2023-07-06T21:41:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd931bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd92c0ca8c0 0x7fd92c0703a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd931bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd92c06c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd931bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c070d20 con 0x7fd92c159220 2023-07-06T21:41:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd931bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c0cad60 con 0x7fd92c0ca8c0 2023-07-06T21:41:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd931bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0cf600 con 0x7fd92c0708e0 2023-07-06T21:41:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd92b7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd92c159220 0x7fd92c0cc3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42798/0 (socket says 172.21.15.84:42798) 2023-07-06T21:41:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3035265841 learned_addr learned my addr 172.21.15.84:0/3035265841 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd923fff700 1 --2- 172.21.15.84:0/3035265841 >> v2:172.21.15.107:3300/0 conn(0x7fd92c0ca8c0 0x7fd92c0703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd92bfff700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd92c06c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643209151 0 0) 0x7fd92c070d20 con 0x7fd92c159220 2023-07-06T21:41:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.559+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9180057c0 con 0x7fd92c159220 2023-07-06T21:41:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585111669 0 0) 0x7fd9180057c0 con 0x7fd92c159220 2023-07-06T21:41:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 msgr2=0x7fd92c06c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:37.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd92c06c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 >> v2:172.21.15.107:3300/0 conn(0x7fd92c0ca8c0 msgr2=0x7fd92c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 --2- 172.21.15.84:0/3035265841 >> v2:172.21.15.107:3300/0 conn(0x7fd92c0ca8c0 0x7fd92c0703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c0ccf40 con 0x7fd92c159220 2023-07-06T21:41:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fd91c001670 con 0x7fd92c159220 2023-07-06T21:41:37.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 >> v1:172.21.15.84:6789/0 conn(0x7fd92c159220 legacy=0x7fd92c0cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd918006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd918006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd918006fa0 0x7fd918007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0cad60 con 0x7fd918005c10 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0cf600 con 0x7fd918006fa0 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91800dc20 con 0x7fd92c0708e0 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd918006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd923fff700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd918006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92bfff700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd918006fa0 0x7fd918007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.560+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 msgr2=0x7fd918006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:37.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd918006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd918006fa0 msgr2=0x7fd918007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd918006fa0 0x7fd918007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3035265841 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9180057c0 con 0x7fd918005c10 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd923fff700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0708e0 0x7fd918006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd918006020 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c190 tx=0x7fd918007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe4b376bff434f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c007b70 con 0x7fd918005c10 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c00ccd0 con 0x7fd918005c10 2023-07-06T21:41:37.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92affd700 1 -- 172.21.15.84:0/3035265841 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c005ba0 con 0x7fd918005c10 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd931bff700 1 -- 172.21.15.84:0/3035265841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 msgr2=0x7fd918006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd931bff700 1 --2- 172.21.15.84:0/3035265841 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd918006020 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c190 tx=0x7fd918007a90 comp rx=0 tx=0).stop 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.561+0000 7fd92bfff700 1 -- 172.21.15.84:0/3035265841 reap_dead start 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.562+0000 7fd931bff700 1 -- 172.21.15.84:0/3035265841 shutdown_connections 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.562+0000 7fd931bff700 1 -- 172.21.15.84:0/3035265841 >> 172.21.15.84:0/3035265841 conn(0x7fd92c0cbf10 msgr2=0x7fd92c0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.562+0000 7fd931bff700 1 -- 172.21.15.84:0/3035265841 shutdown_connections 2023-07-06T21:41:37.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.562+0000 7fd931bff700 1 -- 172.21.15.84:0/3035265841 wait complete. 2023-07-06T21:41:37.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.563+0000 7fd931bff700 1 Processor -- start 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 -- start start 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd92c155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c1e44b0 0x7fd92c153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0c7aa0 0x7fd92c0c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0c8450 con 0x7fd918005c10 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c157ec0 con 0x7fd92c0c7aa0 2023-07-06T21:41:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.564+0000 7fd931bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c157b90 con 0x7fd92c1e44b0 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd92c155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd92c155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34066/0 (socket says 172.21.15.84:34066) 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3584407117 learned_addr learned my addr 172.21.15.84:0/3584407117 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd923fff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c1e44b0 0x7fd92c153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92bfff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0c7aa0 0x7fd92c0c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c1e44b0 msgr2=0x7fd92c153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.575 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c1e44b0 0x7fd92c153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0c7aa0 msgr2=0x7fd92c0c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0c7aa0 0x7fd92c0c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 -- 172.21.15.84:0/3584407117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c155e90 con 0x7fd918005c10 2023-07-06T21:41:37.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92bfff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0c7aa0 0x7fd92c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.565+0000 7fd92b7fe700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd92c155400 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd918006930 tx=0x7fd91c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.566+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0070d0 con 0x7fd918005c10 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.566+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c007230 con 0x7fd918005c10 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.566+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c156170 con 0x7fd918005c10 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.566+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0073c0 con 0x7fd918005c10 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.566+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c1e4920 con 0x7fd918005c10 2023-07-06T21:41:37.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.567+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c070e90 con 0x7fd918005c10 2023-07-06T21:41:37.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.568+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd91c019030 con 0x7fd918005c10 2023-07-06T21:41:37.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.569+0000 7fd928ff9700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd90c06ace0 0x7fd90c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.569+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd91c08a420 con 0x7fd918005c10 2023-07-06T21:41:37.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.569+0000 7fd923fff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd90c06ace0 0x7fd90c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.570+0000 7fd923fff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd90c06ace0 0x7fd90c06d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0ade50 tx=0x7fd914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.573+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd91c055330 con 0x7fd918005c10 2023-07-06T21:41:37.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.769+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd92c1579f0 con 0x7fd918005c10 2023-07-06T21:41:37.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.770+0000 7fd928ff9700 1 -- 172.21.15.84:0/3584407117 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd92c1579f0 con 0x7fd918005c10 2023-07-06T21:41:37.773 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd90c06ace0 msgr2=0x7fd90c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd90c06ace0 0x7fd90c06d190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0ade50 tx=0x7fd914008040 comp rx=0 tx=0).stop 2023-07-06T21:41:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 msgr2=0x7fd92c155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd92c155400 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd918006930 tx=0x7fd91c001930 comp rx=0 tx=0).stop 2023-07-06T21:41:37.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 shutdown_connections 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd90c06ace0 0x7fd90c06d190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0c7aa0 0x7fd92c0c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd918005c10 0x7fd92c155400 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 --2- 172.21.15.84:0/3584407117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c1e44b0 0x7fd92c153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 >> 172.21.15.84:0/3584407117 conn(0x7fd92c0cbf10 msgr2=0x7fd92c0ac3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 shutdown_connections 2023-07-06T21:41:37.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.772+0000 7fd931bff700 1 -- 172.21.15.84:0/3584407117 wait complete. 2023-07-06T21:41:37.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:37.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.987+0000 7fd3017ff700 1 Processor -- start 2023-07-06T21:41:37.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.987+0000 7fd3017ff700 1 -- start start 2023-07-06T21:41:37.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.988+0000 7fd3017ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc069e90 0x7fd2fc06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.988+0000 7fd3017ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2fc0cf280 0x7fd2fc0b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.988+0000 7fd3017ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc0c94b0 con 0x7fd2fc06dde0 2023-07-06T21:41:37.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.988+0000 7fd3017ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc06d350 con 0x7fd2fc0cf280 2023-07-06T21:41:37.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.988+0000 7fd3017ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0ca9b0 con 0x7fd2fc069e90 2023-07-06T21:41:37.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.988+0000 7fd2fb7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2fc0cf280 0x7fd2fc0b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd2fc0cf280 0x7fd2fc0b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34068/0 (socket says 172.21.15.84:34068) 2023-07-06T21:41:37.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/363656393 learned_addr learned my addr 172.21.15.84:0/363656393 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2faffd700 1 --2- 172.21.15.84:0/363656393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc069e90 0x7fd2fc06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/363656393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc069e90 msgr2=0x7fd2fc06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 --2- 172.21.15.84:0/363656393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc069e90 0x7fd2fc06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/363656393 >> v1:172.21.15.84:6789/0 conn(0x7fd2fc06dde0 legacy=0x7fd2fc06a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2f9ffb700 1 -- 172.21.15.84:0/363656393 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460685134 0 0) 0x7fd2fc0c94b0 con 0x7fd2fc06dde0 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/363656393 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0cac90 con 0x7fd2fc0cf280 2023-07-06T21:41:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2fb7fe700 1 --2- 172.21.15.84:0/363656393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc0b1650 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e50 tx=0x7fd2f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7478ad8f6beed19 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.989+0000 7fd2f9ffb700 1 -- 172.21.15.84:0/363656393 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400e070 con 0x7fd2fc0cf280 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd2f9ffb700 1 -- 172.21.15.84:0/363656393 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f400b9c0 con 0x7fd2fc0cf280 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd2f9ffb700 1 -- 172.21.15.84:0/363656393 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400bd70 con 0x7fd2fc0cf280 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 -- 172.21.15.84:0/363656393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 msgr2=0x7fd2fc0b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 --2- 172.21.15.84:0/363656393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc0b1650 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e50 tx=0x7fd2f400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 -- 172.21.15.84:0/363656393 shutdown_connections 2023-07-06T21:41:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 --2- 172.21.15.84:0/363656393 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc0b1650 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 --2- 172.21.15.84:0/363656393 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc069e90 0x7fd2fc06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 -- 172.21.15.84:0/363656393 >> 172.21.15.84:0/363656393 conn(0x7fd2fc0cbf10 msgr2=0x7fd2fc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 -- 172.21.15.84:0/363656393 shutdown_connections 2023-07-06T21:41:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.990+0000 7fd3017ff700 1 -- 172.21.15.84:0/363656393 wait complete. 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.992+0000 7fd3017ff700 1 Processor -- start 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.992+0000 7fd3017ff700 1 -- start start 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd3017ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc069e90 0x7fd2fc150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd3017ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc06dde0 0x7fd2fc0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd3017ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd3017ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c79f0 con 0x7fd2fc0cf280 2023-07-06T21:41:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd3017ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c6d70 con 0x7fd2fc069e90 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd3017ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0c7050 con 0x7fd2fc06dde0 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd2fb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd2fb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc146430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34072/0 (socket says 172.21.15.84:34072) 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/3054954663 learned_addr learned my addr 172.21.15.84:0/3054954663 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd2fa7fc700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc06dde0 0x7fd2fc0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.993+0000 7fd2faffd700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc069e90 0x7fd2fc150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc06dde0 msgr2=0x7fd2fc0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2fb7fe700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc06dde0 0x7fd2fc0c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc069e90 msgr2=0x7fd2fc150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2fb7fe700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc069e90 0x7fd2fc150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2fb7fe700 1 -- 172.21.15.84:0/3054954663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009b30 con 0x7fd2fc0cf280 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2fb7fe700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc146430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b730 tx=0x7fd2f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40048b0 con 0x7fd2fc0cf280 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f40099b0 con 0x7fd2fc0cf280 2023-07-06T21:41:38.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc0c7330 con 0x7fd2fc0cf280 2023-07-06T21:41:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f401c680 con 0x7fd2fc0cf280 2023-07-06T21:41:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.994+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc150f40 con 0x7fd2fc0cf280 2023-07-06T21:41:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.996+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc04e1d0 con 0x7fd2fc0cf280 2023-07-06T21:41:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.996+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd2f401c7e0 con 0x7fd2fc0cf280 2023-07-06T21:41:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.998+0000 7fd2e37fe700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.998+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd2f408b190 con 0x7fd2fc0cf280 2023-07-06T21:41:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.998+0000 7fd2faffd700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.002 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:37.999+0000 7fd2faffd700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd2e806ace0 0x7fd2e806d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec006b20 tx=0x7fd2ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.002+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd2f4056140 con 0x7fd2fc0cf280 2023-07-06T21:41:38.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.201+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2fc15a240 con 0x7fd2fc0cf280 2023-07-06T21:41:38.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.201+0000 7fd2e37fe700 1 -- 172.21.15.84:0/3054954663 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2f4059760 con 0x7fd2fc0cf280 2023-07-06T21:41:38.205 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd2e806ace0 msgr2=0x7fd2e806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd2e806ace0 0x7fd2e806d190 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec006b20 tx=0x7fd2ec008040 comp rx=0 tx=0).stop 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 msgr2=0x7fd2fc146430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc146430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400b730 tx=0x7fd2f4004ad0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 shutdown_connections 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd2e806ace0 0x7fd2e806d190 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2fc0cf280 0x7fd2fc146430 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2fc069e90 0x7fd2fc150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 --2- 172.21.15.84:0/3054954663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2fc06dde0 0x7fd2fc0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.203+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 >> 172.21.15.84:0/3054954663 conn(0x7fd2fc0cbf10 msgr2=0x7fd2fc0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:38.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.204+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 shutdown_connections 2023-07-06T21:41:38.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.204+0000 7fd3017ff700 1 -- 172.21.15.84:0/3054954663 wait complete. 2023-07-06T21:41:38.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:38.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.418+0000 7f0170fff700 1 Processor -- start 2023-07-06T21:41:38.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.418+0000 7f0170fff700 1 -- start start 2023-07-06T21:41:38.424 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.419+0000 7f0170fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f016c0b81c0 0x7f016c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.419+0000 7f0170fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.419+0000 7f0170fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f016c0ceda0 0x7f016c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.419+0000 7f0170fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0d1830 con 0x7f016c0ceda0 2023-07-06T21:41:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.419+0000 7f0170fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f016c0cf330 con 0x7f016c0b81c0 2023-07-06T21:41:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.419+0000 7f0170fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c0d1270 con 0x7f016c0b1a80 2023-07-06T21:41:38.425 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f016c0ceda0 0x7f016c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f016c0ceda0 0x7f016c15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34086/0 (socket says 172.21.15.84:34086) 2023-07-06T21:41:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 -- 172.21.15.84:0/2689694407 learned_addr learned my addr 172.21.15.84:0/2689694407 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f0163fff700 1 --2- 172.21.15.84:0/2689694407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016a59c700 1 --2- 172.21.15.84:0/2689694407 >> v2:172.21.15.107:3300/0 conn(0x7f016c0b81c0 0x7f016c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 -- 172.21.15.84:0/2689694407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 msgr2=0x7f016c0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:38.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 --2- 172.21.15.84:0/2689694407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 -- 172.21.15.84:0/2689694407 >> v2:172.21.15.107:3300/0 conn(0x7f016c0b81c0 msgr2=0x7f016c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:38.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 --2- 172.21.15.84:0/2689694407 >> v2:172.21.15.107:3300/0 conn(0x7f016c0b81c0 0x7f016c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 -- 172.21.15.84:0/2689694407 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016c0d1550 con 0x7f016c0ceda0 2023-07-06T21:41:38.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016a59c700 1 --2- 172.21.15.84:0/2689694407 >> v2:172.21.15.107:3300/0 conn(0x7f016c0b81c0 0x7f016c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:38.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.420+0000 7f016ad9d700 1 --2- 172.21.15.84:0/2689694407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0ceda0 0x7f016c15ff20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f016c0b2620 tx=0x7f016400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=384a5f4654415aeb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0169d9b700 1 -- 172.21.15.84:0/2689694407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400e070 con 0x7f016c0ceda0 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0169d9b700 1 -- 172.21.15.84:0/2689694407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016400b9c0 con 0x7f016c0ceda0 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0169d9b700 1 -- 172.21.15.84:0/2689694407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016400bd70 con 0x7f016c0ceda0 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 -- 172.21.15.84:0/2689694407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0ceda0 msgr2=0x7f016c15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 --2- 172.21.15.84:0/2689694407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0ceda0 0x7f016c15ff20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f016c0b2620 tx=0x7f016400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 -- 172.21.15.84:0/2689694407 shutdown_connections 2023-07-06T21:41:38.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 --2- 172.21.15.84:0/2689694407 >> v2:172.21.15.107:3300/0 conn(0x7f016c0b81c0 0x7f016c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 --2- 172.21.15.84:0/2689694407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0ceda0 0x7f016c15ff20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 --2- 172.21.15.84:0/2689694407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 -- 172.21.15.84:0/2689694407 >> 172.21.15.84:0/2689694407 conn(0x7f016c06b2b0 msgr2=0x7f016c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:38.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 -- 172.21.15.84:0/2689694407 shutdown_connections 2023-07-06T21:41:38.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.421+0000 7f0170fff700 1 -- 172.21.15.84:0/2689694407 wait complete. 2023-07-06T21:41:38.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.423+0000 7f0170fff700 1 Processor -- start 2023-07-06T21:41:38.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.423+0000 7f0170fff700 1 -- start start 2023-07-06T21:41:38.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0170fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0170fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 0x7f016c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0170fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f016c0ceda0 0x7f016c154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0170fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c1e50f0 con 0x7f016c0b81c0 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0170fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016c154aa0 con 0x7f016c0ceda0 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0170fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016c158ec0 con 0x7f016c0b1a80 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0163fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 0x7f016c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0163fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 0x7f016c158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34096/0 (socket says 172.21.15.84:34096) 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.424+0000 7f0163fff700 1 -- 172.21.15.84:0/3669714869 learned_addr learned my addr 172.21.15.84:0/3669714869 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:38.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f016a59c700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f016ad9d700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f016c0ceda0 0x7f016c154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0163fff700 1 -- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 msgr2=0x7f016c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0163fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c154cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0163fff700 1 -- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f016c0ceda0 msgr2=0x7f016c154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0163fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f016c0ceda0 0x7f016c154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0163fff700 1 -- 172.21.15.84:0/3669714869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0164009b30 con 0x7f016c0b81c0 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0163fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 0x7f016c158870 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f015400e810 tx=0x7f015400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f015400c070 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154013610 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540139c0 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.425+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016c1e5390 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.426+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016c1e5950 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.427+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f016c04e1d0 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.427+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0154013b20 con 0x7f016c0b81c0 2023-07-06T21:41:38.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.429+0000 7f0162ffd700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f014c06ace0 0x7f014c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.429+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f01540893e0 con 0x7f016c0b81c0 2023-07-06T21:41:38.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.429+0000 7f016a59c700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f014c06ace0 0x7f014c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.430+0000 7f016a59c700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f014c06ace0 0x7f014c06d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f016c15c720 tx=0x7f015c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.433+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0154054270 con 0x7f016c0b81c0 2023-07-06T21:41:38.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.628+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f016c0ce6a0 con 0x7f016c0b81c0 2023-07-06T21:41:38.632 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.629+0000 7f0162ffd700 1 -- 172.21.15.84:0/3669714869 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0154057890 con 0x7f016c0b81c0 2023-07-06T21:41:38.633 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.631+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f014c06ace0 msgr2=0x7f014c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.631+0000 7f0170fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f014c06ace0 0x7f014c06d190 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f016c15c720 tx=0x7f015c008040 comp rx=0 tx=0).stop 2023-07-06T21:41:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 msgr2=0x7f016c158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 0x7f016c158870 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f015400e810 tx=0x7f015400eb20 comp rx=0 tx=0).stop 2023-07-06T21:41:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 shutdown_connections 2023-07-06T21:41:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f014c06ace0 0x7f014c06d190 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f016c0ceda0 0x7f016c154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f016c0b1a80 0x7f016c154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 --2- 172.21.15.84:0/3669714869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f016c0b81c0 0x7f016c158870 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 >> 172.21.15.84:0/3669714869 conn(0x7f016c06b2b0 msgr2=0x7f016c0c9510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 shutdown_connections 2023-07-06T21:41:38.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.632+0000 7f0170fff700 1 -- 172.21.15.84:0/3669714869 wait complete. 2023-07-06T21:41:38.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:38.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.846+0000 7f6a9adff700 1 Processor -- start 2023-07-06T21:41:38.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.846+0000 7f6a9adff700 1 -- start start 2023-07-06T21:41:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a9adff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a940cfd10 0x7f6a940d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a9adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940b1a80 0x7f6a940d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a9adff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a940d1830 con 0x7f6a940b81c0 2023-07-06T21:41:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a9adff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a940cf330 con 0x7f6a940cfd10 2023-07-06T21:41:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a9adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a940d1270 con 0x7f6a940b1a80 2023-07-06T21:41:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a93fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a940cfd10 0x7f6a940d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a93fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a940cfd10 0x7f6a940d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34108/0 (socket says 172.21.15.84:34108) 2023-07-06T21:41:38.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.847+0000 7f6a93fff700 1 -- 172.21.15.84:0/3975977564 learned_addr learned my addr 172.21.15.84:0/3975977564 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:38.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a9939c700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940b1a80 0x7f6a940d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a93fff700 1 -- 172.21.15.84:0/3975977564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940b1a80 msgr2=0x7f6a940d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:38.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a93fff700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940b1a80 0x7f6a940d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a93fff700 1 -- 172.21.15.84:0/3975977564 >> v1:172.21.15.107:6789/0 conn(0x7f6a940b81c0 legacy=0x7f6a940cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:38.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a93fff700 1 -- 172.21.15.84:0/3975977564 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940d1550 con 0x7f6a940cfd10 2023-07-06T21:41:38.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a9939c700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940b1a80 0x7f6a940d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:38.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a93fff700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940cfd10 0x7f6a940d2380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a800098f0 tx=0x7f6a80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf47737d7af80e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a937fe700 1 -- 172.21.15.84:0/3975977564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a80004230 con 0x7f6a940cfd10 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a937fe700 1 -- 172.21.15.84:0/3975977564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a80004390 con 0x7f6a940cfd10 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.848+0000 7f6a937fe700 1 -- 172.21.15.84:0/3975977564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a800036a0 con 0x7f6a940cfd10 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 -- 172.21.15.84:0/3975977564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940cfd10 msgr2=0x7f6a940d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940cfd10 0x7f6a940d2380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6a800098f0 tx=0x7f6a80009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 -- 172.21.15.84:0/3975977564 shutdown_connections 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940b1a80 0x7f6a940d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 --2- 172.21.15.84:0/3975977564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940cfd10 0x7f6a940d2380 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 -- 172.21.15.84:0/3975977564 >> 172.21.15.84:0/3975977564 conn(0x7f6a9406b2b0 msgr2=0x7f6a9406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:38.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 -- 172.21.15.84:0/3975977564 shutdown_connections 2023-07-06T21:41:38.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.849+0000 7f6a9adff700 1 -- 172.21.15.84:0/3975977564 wait complete. 2023-07-06T21:41:38.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.851+0000 7f6a9adff700 1 Processor -- start 2023-07-06T21:41:38.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.851+0000 7f6a9adff700 1 -- start start 2023-07-06T21:41:38.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9adff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a940b1a80 0x7f6a9415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9adff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 0x7f6a9415ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940cfd10 0x7f6a941495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9adff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a941491d0 con 0x7f6a940b81c0 2023-07-06T21:41:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9adff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9415fc90 con 0x7f6a940b1a80 2023-07-06T21:41:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9415d2b0 con 0x7f6a940cfd10 2023-07-06T21:41:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 0x7f6a9415ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 0x7f6a9415ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34110/0 (socket says 172.21.15.84:34110) 2023-07-06T21:41:38.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 -- 172.21.15.84:0/470684344 learned_addr learned my addr 172.21.15.84:0/470684344 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a9939c700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940cfd10 0x7f6a941495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a98b9b700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a940b1a80 0x7f6a9415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 -- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940cfd10 msgr2=0x7f6a941495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940cfd10 0x7f6a941495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 -- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a940b1a80 msgr2=0x7f6a9415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:38.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a940b1a80 0x7f6a9415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.853+0000 7f6a93fff700 1 -- 172.21.15.84:0/470684344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a80009550 con 0x7f6a940b81c0 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.854+0000 7f6a93fff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 0x7f6a9415ca20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a80004000 tx=0x7f6a80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.854+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000b860 con 0x7f6a940b81c0 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.854+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8000b9e0 con 0x7f6a940b81c0 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.854+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9415d590 con 0x7f6a940b81c0 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.854+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000bd90 con 0x7f6a940b81c0 2023-07-06T21:41:38.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.854+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a94148df0 con 0x7f6a940b81c0 2023-07-06T21:41:38.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.855+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a94148f80 con 0x7f6a940b81c0 2023-07-06T21:41:38.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.856+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a8001b440 con 0x7f6a940b81c0 2023-07-06T21:41:38.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.857+0000 7f6a917fa700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a7806ace0 0x7f6a7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:38.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.857+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6a8008af20 con 0x7f6a940b81c0 2023-07-06T21:41:38.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.857+0000 7f6a98b9b700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a7806ace0 0x7f6a7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:38.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.858+0000 7f6a98b9b700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a7806ace0 0x7f6a7806d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004f30 tx=0x7f6a88009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:38.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:38.861+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6a80055e30 con 0x7f6a940b81c0 2023-07-06T21:41:39.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.058+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f6a9404e1d0 con 0x7f6a940b81c0 2023-07-06T21:41:39.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.058+0000 7f6a917fa700 1 -- 172.21.15.84:0/470684344 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7f6a80059450 con 0x7f6a940b81c0 2023-07-06T21:41:39.062 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:39.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a7806ace0 msgr2=0x7f6a7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a7806ace0 0x7f6a7806d190 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004f30 tx=0x7f6a88009040 comp rx=0 tx=0).stop 2023-07-06T21:41:39.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 msgr2=0x7f6a9415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 0x7f6a9415ca20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6a80004000 tx=0x7f6a80004750 comp rx=0 tx=0).stop 2023-07-06T21:41:39.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 shutdown_connections 2023-07-06T21:41:39.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a7806ace0 0x7f6a7806d190 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a940cfd10 0x7f6a941495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a940b1a80 0x7f6a9415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 --2- 172.21.15.84:0/470684344 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a940b81c0 0x7f6a9415ca20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 >> 172.21.15.84:0/470684344 conn(0x7f6a9406b2b0 msgr2=0x7f6a940d2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:39.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 shutdown_connections 2023-07-06T21:41:39.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.061+0000 7f6a9adff700 1 -- 172.21.15.84:0/470684344 wait complete. 2023-07-06T21:41:39.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:41:39.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.275+0000 7f7b3f7ff700 1 Processor -- start 2023-07-06T21:41:39.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.275+0000 7f7b3f7ff700 1 -- start start 2023-07-06T21:41:39.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.276+0000 7f7b3f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b38154910 0x7f7b38070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.276+0000 7f7b3f7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b3815a7e0 0x7f7b3815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.276+0000 7f7b3f7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b38070230 con 0x7f7b380c8d00 2023-07-06T21:41:39.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.276+0000 7f7b3f7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3815b2a0 con 0x7f7b3815a7e0 2023-07-06T21:41:39.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.276+0000 7f7b3f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3815b580 con 0x7f7b38154910 2023-07-06T21:41:39.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3cd9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7b3815a7e0 0x7f7b3815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3dd9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7b380c8d00 0x7f7b380c3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42816/0 (socket says 172.21.15.84:42816) 2023-07-06T21:41:39.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3dd9c700 1 -- 172.21.15.84:0/3843914754 learned_addr learned my addr 172.21.15.84:0/3843914754 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3d59b700 1 --2- 172.21.15.84:0/3843914754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b38154910 0x7f7b38070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3cd9a700 1 -- 172.21.15.84:0/3843914754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b38154910 msgr2=0x7f7b38070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3cd9a700 1 --2- 172.21.15.84:0/3843914754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b38154910 0x7f7b38070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3cd9a700 1 -- 172.21.15.84:0/3843914754 >> v1:172.21.15.84:6789/0 conn(0x7f7b380c8d00 legacy=0x7f7b380c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3cd9a700 1 -- 172.21.15.84:0/3843914754 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b38070480 con 0x7f7b3815a7e0 2023-07-06T21:41:39.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b2ffff700 1 -- 172.21.15.84:0/3843914754 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337553574 0 0) 0x7f7b38070230 con 0x7f7b380c8d00 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.277+0000 7f7b3cd9a700 1 --2- 172.21.15.84:0/3843914754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3815a7e0 0x7f7b3815ac50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009840 tx=0x7f7b28009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a17c2c8338c4a78 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b2ffff700 1 -- 172.21.15.84:0/3843914754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28011830 con 0x7f7b3815a7e0 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b2ffff700 1 -- 172.21.15.84:0/3843914754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b28011990 con 0x7f7b3815a7e0 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b2ffff700 1 -- 172.21.15.84:0/3843914754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28011d20 con 0x7f7b3815a7e0 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/3843914754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3815a7e0 msgr2=0x7f7b3815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/3843914754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3815a7e0 0x7f7b3815ac50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009840 tx=0x7f7b28009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:39.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/3843914754 shutdown_connections 2023-07-06T21:41:39.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/3843914754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b38154910 0x7f7b38070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/3843914754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b3815a7e0 0x7f7b3815ac50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/3843914754 >> 172.21.15.84:0/3843914754 conn(0x7f7b380cf3d0 msgr2=0x7f7b380cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:39.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.278+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/3843914754 shutdown_connections 2023-07-06T21:41:39.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.279+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/3843914754 wait complete. 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.280+0000 7f7b3f7ff700 1 Processor -- start 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.280+0000 7f7b3f7ff700 1 -- start start 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3f7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 0x7f7b3813f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3f7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38154910 0x7f7b38153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b3815a7e0 0x7f7b38153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3f7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381f5ce0 con 0x7f7b380c8d00 2023-07-06T21:41:39.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3f7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380b1260 con 0x7f7b38154910 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381f6100 con 0x7f7b3815a7e0 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.281+0000 7f7b3d59b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 0x7f7b3813f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 0x7f7b3813f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34126/0 (socket says 172.21.15.84:34126) 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 -- 172.21.15.84:0/1926667865 learned_addr learned my addr 172.21.15.84:0/1926667865 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3dd9c700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b3815a7e0 0x7f7b38153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3cd9a700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38154910 0x7f7b38153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 -- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b3815a7e0 msgr2=0x7f7b38153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b3815a7e0 0x7f7b38153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 -- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38154910 msgr2=0x7f7b38153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38154910 0x7f7b38153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 -- 172.21.15.84:0/1926667865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b280094a0 con 0x7f7b380c8d00 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3d59b700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 0x7f7b3813f170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400e9e0 tx=0x7f7b3400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b34009710 con 0x7f7b380c8d00 2023-07-06T21:41:39.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3400c7f0 con 0x7f7b380c8d00 2023-07-06T21:41:39.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3813f6b0 con 0x7f7b380c8d00 2023-07-06T21:41:39.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3400cb60 con 0x7f7b380c8d00 2023-07-06T21:41:39.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.282+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b381f5620 con 0x7f7b380c8d00 2023-07-06T21:41:39.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.283+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3804e240 con 0x7f7b380c8d00 2023-07-06T21:41:39.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.284+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7b3400ccc0 con 0x7f7b380c8d00 2023-07-06T21:41:39.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.286+0000 7f7b2dffb700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7b1c06ace0 0x7f7b1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.286+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f7b34089660 con 0x7f7b380c8d00 2023-07-06T21:41:39.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.286+0000 7f7b3cd9a700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7b1c06ace0 0x7f7b1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.287+0000 7f7b3cd9a700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7b1c06ace0 0x7f7b1c06d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b280090b0 tx=0x7f7b2801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:39.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.290+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7b34054610 con 0x7f7b380c8d00 2023-07-06T21:41:39.507 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.504+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b380cbbe0 con 0x7f7b380c8d00 2023-07-06T21:41:39.508 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.505+0000 7f7b2dffb700 1 -- 172.21.15.84:0/1926667865 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7b34057c30 con 0x7f7b380c8d00 2023-07-06T21:41:39.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7b1c06ace0 msgr2=0x7f7b1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7b1c06ace0 0x7f7b1c06d190 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b280090b0 tx=0x7f7b2801c040 comp rx=0 tx=0).stop 2023-07-06T21:41:39.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 msgr2=0x7f7b3813f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 0x7f7b3813f170 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7b3400e9e0 tx=0x7f7b3400eda0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 shutdown_connections 2023-07-06T21:41:39.511 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7b1c06ace0 0x7f7b1c06d190 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b3815a7e0 0x7f7b38153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7b380c8d00 0x7f7b3813f170 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 --2- 172.21.15.84:0/1926667865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b38154910 0x7f7b38153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 >> 172.21.15.84:0/1926667865 conn(0x7f7b380cf3d0 msgr2=0x7f7b38152020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:39.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 shutdown_connections 2023-07-06T21:41:39.512 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.507+0000 7f7b3f7ff700 1 -- 172.21.15.84:0/1926667865 wait complete. 2023-07-06T21:41:39.513 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:41:39.529 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:39.529 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:39.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.802+0000 7f1a649ff700 1 Processor -- start 2023-07-06T21:41:39.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.802+0000 7f1a649ff700 1 -- start start 2023-07-06T21:41:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a649ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a600d3710 0x7f1a600cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a649ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a600cd910 con 0x7f1a600cfa90 2023-07-06T21:41:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a649ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a600d0a80 con 0x7f1a600b83c0 2023-07-06T21:41:39.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a649ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600cff10 con 0x7f1a600d3710 2023-07-06T21:41:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a57fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a600d3710 0x7f1a600cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a57fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1a600d3710 0x7f1a600cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34132/0 (socket says 172.21.15.84:34132) 2023-07-06T21:41:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a57fff700 1 -- 172.21.15.84:0/565081839 learned_addr learned my addr 172.21.15.84:0/565081839 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a5ed9d700 1 --1- 172.21.15.84:0/565081839 >> v1:172.21.15.84:6789/0 conn(0x7f1a600cfa90 0x7f1a600d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42832/0 (socket says 172.21.15.84:42832) 2023-07-06T21:41:39.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.803+0000 7f1a57fff700 1 -- 172.21.15.84:0/565081839 >> v1:172.21.15.107:6789/0 conn(0x7f1a600b83c0 legacy=0x7f1a600cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:39.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.804+0000 7f1a57fff700 1 -- 172.21.15.84:0/565081839 >> v1:172.21.15.84:6789/0 conn(0x7f1a600cfa90 legacy=0x7f1a600d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.804+0000 7f1a57fff700 1 -- 172.21.15.84:0/565081839 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600b8790 con 0x7f1a600d3710 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.804+0000 7f1a57fff700 1 --2- 172.21.15.84:0/565081839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600cf550 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098f0 tx=0x7f1a48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ada49cc96ef9d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.804+0000 7f1a5dd9b700 1 -- 172.21.15.84:0/565081839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48004230 con 0x7f1a600d3710 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.804+0000 7f1a5dd9b700 1 -- 172.21.15.84:0/565081839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48004390 con 0x7f1a600d3710 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.804+0000 7f1a5dd9b700 1 -- 172.21.15.84:0/565081839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a480036a0 con 0x7f1a600d3710 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 -- 172.21.15.84:0/565081839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 msgr2=0x7f1a600cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 --2- 172.21.15.84:0/565081839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600cf550 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1a480098f0 tx=0x7f1a48009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:39.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 -- 172.21.15.84:0/565081839 shutdown_connections 2023-07-06T21:41:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 --2- 172.21.15.84:0/565081839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600cf550 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 -- 172.21.15.84:0/565081839 >> 172.21.15.84:0/565081839 conn(0x7f1a6006b350 msgr2=0x7f1a6006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 -- 172.21.15.84:0/565081839 shutdown_connections 2023-07-06T21:41:39.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.805+0000 7f1a649ff700 1 -- 172.21.15.84:0/565081839 wait complete. 2023-07-06T21:41:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.806+0000 7f1a649ff700 1 Processor -- start 2023-07-06T21:41:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.807+0000 7f1a649ff700 1 -- start start 2023-07-06T21:41:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a649ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a600b83c0 0x7f1a6015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a649ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a600cfa90 0x7f1a600c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a649ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a649ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6015f7a0 con 0x7f1a600d3710 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a649ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c6fe0 con 0x7f1a600b83c0 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a649ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600c72c0 con 0x7f1a600cfa90 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34142/0 (socket says 172.21.15.84:34142) 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 -- 172.21.15.84:0/528560757 learned_addr learned my addr 172.21.15.84:0/528560757 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a57fff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a600cfa90 0x7f1a600c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5e59c700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a600b83c0 0x7f1a6015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 -- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a600cfa90 msgr2=0x7f1a600c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a600cfa90 0x7f1a600c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 -- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a600b83c0 msgr2=0x7f1a6015f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a600b83c0 0x7f1a6015f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5ed9d700 1 -- 172.21.15.84:0/528560757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a48009550 con 0x7f1a600d3710 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a5e59c700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a600b83c0 0x7f1a6015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:39.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.808+0000 7f1a57fff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a600cfa90 0x7f1a600c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.809+0000 7f1a5ed9d700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600c6670 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800cbc0 tx=0x7f1a5800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.809+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800ab50 con 0x7f1a600d3710 2023-07-06T21:41:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.809+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800acb0 con 0x7f1a600d3710 2023-07-06T21:41:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.809+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a600c75a0 con 0x7f1a600d3710 2023-07-06T21:41:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.809+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58007270 con 0x7f1a600d3710 2023-07-06T21:41:39.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.809+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a600c7b60 con 0x7f1a600d3710 2023-07-06T21:41:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.810+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a600677b0 con 0x7f1a600d3710 2023-07-06T21:41:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.811+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1a580073d0 con 0x7f1a600d3710 2023-07-06T21:41:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.812+0000 7f1a56ffd700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1a4006ace0 0x7f1a4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:39.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.812+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f1a58089160 con 0x7f1a600d3710 2023-07-06T21:41:39.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.812+0000 7f1a5e59c700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1a4006ace0 0x7f1a4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:39.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.813+0000 7f1a5e59c700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1a4006ace0 0x7f1a4006d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006fd0 tx=0x7f1a50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:39.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:39.816+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f1a58054130 con 0x7f1a600d3710 2023-07-06T21:41:40.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.014+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a600cd910 con 0x7f1a600d3710 2023-07-06T21:41:40.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.016+0000 7f1a56ffd700 1 -- 172.21.15.84:0/528560757 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7f1a600cd910 con 0x7f1a600d3710 2023-07-06T21:41:40.019 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:40.019 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1a4006ace0 msgr2=0x7f1a4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1a4006ace0 0x7f1a4006d190 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006fd0 tx=0x7f1a50008040 comp rx=0 tx=0).stop 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 msgr2=0x7f1a600c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600c6670 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800cbc0 tx=0x7f1a5800cf80 comp rx=0 tx=0).stop 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 shutdown_connections 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f1a4006ace0 0x7f1a4006d190 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1a600d3710 0x7f1a600c6670 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a600b83c0 0x7f1a6015f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 --2- 172.21.15.84:0/528560757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a600cfa90 0x7f1a600c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 >> 172.21.15.84:0/528560757 conn(0x7f1a6006b350 msgr2=0x7f1a600d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:40.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 shutdown_connections 2023-07-06T21:41:40.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.018+0000 7f1a649ff700 1 -- 172.21.15.84:0/528560757 wait complete. 2023-07-06T21:41:40.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:40.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:40.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.235+0000 7f94f3dff700 1 Processor -- start 2023-07-06T21:41:40.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.235+0000 7f94f3dff700 1 -- start start 2023-07-06T21:41:40.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.236+0000 7f94f3dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f94ec0b81c0 0x7f94ec0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.236+0000 7f94f3dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 0x7f94ec0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.236+0000 7f94f3dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0d1830 con 0x7f94ec0ceda0 2023-07-06T21:41:40.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.236+0000 7f94f3dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec0cf330 con 0x7f94ec0b81c0 2023-07-06T21:41:40.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.236+0000 7f94f3dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0d1270 con 0x7f94ec0b1a80 2023-07-06T21:41:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 0x7f94ec0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f239c700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f94ec0ceda0 0x7f94ec15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:48182/0 (socket says 172.21.15.84:48182) 2023-07-06T21:41:40.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f239c700 1 -- 172.21.15.84:0/2733853240 learned_addr learned my addr 172.21.15.84:0/2733853240 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f1b9b700 1 --2- 172.21.15.84:0/2733853240 >> v2:172.21.15.107:3300/0 conn(0x7f94ec0b81c0 0x7f94ec0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 0x7f94ec0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:52862/0 (socket says 172.21.15.84:52862) 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f0b99700 1 -- 172.21.15.84:0/2733853240 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807011509 0 0) 0x7f94ec0d1830 con 0x7f94ec0ceda0 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 -- 172.21.15.84:0/2733853240 >> v2:172.21.15.107:3300/0 conn(0x7f94ec0b81c0 msgr2=0x7f94ec0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 --2- 172.21.15.84:0/2733853240 >> v2:172.21.15.107:3300/0 conn(0x7f94ec0b81c0 0x7f94ec0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 -- 172.21.15.84:0/2733853240 >> v1:172.21.15.107:6789/0 conn(0x7f94ec0ceda0 legacy=0x7f94ec15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 -- 172.21.15.84:0/2733853240 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec0d1550 con 0x7f94ec0b1a80 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.237+0000 7f94f139a700 1 --2- 172.21.15.84:0/2733853240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 0x7f94ec0cfb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009840 tx=0x7f94dc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1af29a6def5f742 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f0b99700 1 -- 172.21.15.84:0/2733853240 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc011830 con 0x7f94ec0b1a80 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f0b99700 1 -- 172.21.15.84:0/2733853240 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc011990 con 0x7f94ec0b1a80 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f0b99700 1 -- 172.21.15.84:0/2733853240 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc011d20 con 0x7f94ec0b1a80 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 -- 172.21.15.84:0/2733853240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 msgr2=0x7f94ec0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 --2- 172.21.15.84:0/2733853240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 0x7f94ec0cfb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009840 tx=0x7f94dc009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 -- 172.21.15.84:0/2733853240 shutdown_connections 2023-07-06T21:41:40.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 --2- 172.21.15.84:0/2733853240 >> v2:172.21.15.107:3300/0 conn(0x7f94ec0b81c0 0x7f94ec0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 --2- 172.21.15.84:0/2733853240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b1a80 0x7f94ec0cfb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 -- 172.21.15.84:0/2733853240 >> 172.21.15.84:0/2733853240 conn(0x7f94ec06b2b0 msgr2=0x7f94ec06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 -- 172.21.15.84:0/2733853240 shutdown_connections 2023-07-06T21:41:40.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.238+0000 7f94f3dff700 1 -- 172.21.15.84:0/2733853240 wait complete. 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.240+0000 7f94f3dff700 1 Processor -- start 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.240+0000 7f94f3dff700 1 -- start start 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.241+0000 7f94f3dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94ec0b1a80 0x7f94ec0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.241+0000 7f94f3dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b81c0 0x7f94ec0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.241+0000 7f94f3dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 0x7f94ec1e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.241+0000 7f94f3dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0c7190 con 0x7f94ec0ceda0 2023-07-06T21:41:40.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.241+0000 7f94f3dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec146290 con 0x7f94ec0b1a80 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.241+0000 7f94f3dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0c8ec0 con 0x7f94ec0b81c0 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 0x7f94ec1e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 0x7f94ec1e5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34144/0 (socket says 172.21.15.84:34144) 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 -- 172.21.15.84:0/3945863050 learned_addr learned my addr 172.21.15.84:0/3945863050 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f139a700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b81c0 0x7f94ec0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f1b9b700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94ec0b1a80 0x7f94ec0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 -- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b81c0 msgr2=0x7f94ec0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b81c0 0x7f94ec0c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 -- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94ec0b1a80 msgr2=0x7f94ec0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94ec0b1a80 0x7f94ec0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 -- 172.21.15.84:0/3945863050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94dc0094a0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94f239c700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 0x7f94ec1e5d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f94e400cba0 tx=0x7f94e400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.242+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400aa40 con 0x7f94ec0ceda0 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.243+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400abc0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.243+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e40070d0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.243+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec0c91a0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.243+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec1464e0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.244+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec1466d0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.245+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f94e4007230 con 0x7f94ec0ceda0 2023-07-06T21:41:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.246+0000 7f94e27fc700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f94d006ace0 0x7f94d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.246+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f94e4088ff0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.246+0000 7f94f1b9b700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f94d006ace0 0x7f94d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.247+0000 7f94f1b9b700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f94d006ace0 0x7f94d006d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009be0 tx=0x7f94e8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:40.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.250+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f94e40bb310 con 0x7f94ec0ceda0 2023-07-06T21:41:40.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.445+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94ec0d3360 con 0x7f94ec0ceda0 2023-07-06T21:41:40.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.445+0000 7f94e27fc700 1 -- 172.21.15.84:0/3945863050 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94e40180a0 con 0x7f94ec0ceda0 2023-07-06T21:41:40.449 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f94d006ace0 msgr2=0x7f94d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f94d006ace0 0x7f94d006d190 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f94e8009be0 tx=0x7f94e8009210 comp rx=0 tx=0).stop 2023-07-06T21:41:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 msgr2=0x7f94ec1e5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 0x7f94ec1e5d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f94e400cba0 tx=0x7f94e400ceb0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 shutdown_connections 2023-07-06T21:41:40.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f94d006ace0 0x7f94d006d190 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94ec0ceda0 0x7f94ec1e5d80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94ec0b1a80 0x7f94ec0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 --2- 172.21.15.84:0/3945863050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94ec0b81c0 0x7f94ec0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 >> 172.21.15.84:0/3945863050 conn(0x7f94ec06b2b0 msgr2=0x7f94ec0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 shutdown_connections 2023-07-06T21:41:40.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.448+0000 7f94f3dff700 1 -- 172.21.15.84:0/3945863050 wait complete. 2023-07-06T21:41:40.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.662+0000 7fba4d3ff700 1 Processor -- start 2023-07-06T21:41:40.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.662+0000 7fba4d3ff700 1 -- start start 2023-07-06T21:41:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.663+0000 7fba4d3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fba480b81c0 0x7fba480cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.663+0000 7fba4d3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480b1a80 0x7fba480cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.663+0000 7fba4d3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba480d1830 con 0x7fba480ceda0 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.663+0000 7fba4d3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba480cf330 con 0x7fba480b81c0 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.663+0000 7fba4d3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba480d1270 con 0x7fba480b1a80 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba467fc700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480b1a80 0x7fba480cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba477fe700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fba480ceda0 0x7fba4815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:48184/0 (socket says 172.21.15.84:48184) 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba477fe700 1 -- 172.21.15.84:0/4147388607 learned_addr learned my addr 172.21.15.84:0/4147388607 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:40.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba46ffd700 1 --2- 172.21.15.84:0/4147388607 >> v2:172.21.15.107:3300/0 conn(0x7fba480b81c0 0x7fba480cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba467fc700 1 -- 172.21.15.84:0/4147388607 >> v2:172.21.15.107:3300/0 conn(0x7fba480b81c0 msgr2=0x7fba480cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba467fc700 1 --2- 172.21.15.84:0/4147388607 >> v2:172.21.15.107:3300/0 conn(0x7fba480b81c0 0x7fba480cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba467fc700 1 -- 172.21.15.84:0/4147388607 >> v1:172.21.15.107:6789/0 conn(0x7fba480ceda0 legacy=0x7fba4815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba467fc700 1 -- 172.21.15.84:0/4147388607 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba480d1550 con 0x7fba480b1a80 2023-07-06T21:41:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba45ffb700 1 -- 172.21.15.84:0/4147388607 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823465731 0 0) 0x7fba480d1830 con 0x7fba480ceda0 2023-07-06T21:41:40.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.664+0000 7fba467fc700 1 --2- 172.21.15.84:0/4147388607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480b1a80 0x7fba480cfb40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009840 tx=0x7fba3c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a23af77911b12793 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba45ffb700 1 -- 172.21.15.84:0/4147388607 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c011830 con 0x7fba480b1a80 2023-07-06T21:41:40.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba45ffb700 1 -- 172.21.15.84:0/4147388607 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c011990 con 0x7fba480b1a80 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba45ffb700 1 -- 172.21.15.84:0/4147388607 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c011d20 con 0x7fba480b1a80 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 -- 172.21.15.84:0/4147388607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480b1a80 msgr2=0x7fba480cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/4147388607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480b1a80 0x7fba480cfb40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009840 tx=0x7fba3c009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 -- 172.21.15.84:0/4147388607 shutdown_connections 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/4147388607 >> v2:172.21.15.107:3300/0 conn(0x7fba480b81c0 0x7fba480cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/4147388607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480b1a80 0x7fba480cfb40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 -- 172.21.15.84:0/4147388607 >> 172.21.15.84:0/4147388607 conn(0x7fba4806b2b0 msgr2=0x7fba4806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.665+0000 7fba4d3ff700 1 -- 172.21.15.84:0/4147388607 shutdown_connections 2023-07-06T21:41:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.666+0000 7fba4d3ff700 1 -- 172.21.15.84:0/4147388607 wait complete. 2023-07-06T21:41:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.667+0000 7fba4d3ff700 1 Processor -- start 2023-07-06T21:41:40.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.667+0000 7fba4d3ff700 1 -- start start 2023-07-06T21:41:40.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba4d3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 0x7fba480c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba4d3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba480b81c0 0x7fba480c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba4d3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480ceda0 0x7fba481e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba4d3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba480c7190 con 0x7fba480b1a80 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba4d3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba48146290 con 0x7fba480b81c0 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba4d3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba480c8ec0 con 0x7fba480ceda0 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.668+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 0x7fba480c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 0x7fba480c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34148/0 (socket says 172.21.15.84:34148) 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 -- 172.21.15.84:0/2896067454 learned_addr learned my addr 172.21.15.84:0/2896067454 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba467fc700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba480b81c0 0x7fba480c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba477fe700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480ceda0 0x7fba481e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 -- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480ceda0 msgr2=0x7fba481e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480ceda0 0x7fba481e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 -- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba480b81c0 msgr2=0x7fba480c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba480b81c0 0x7fba480c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 -- 172.21.15.84:0/2896067454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c0094a0 con 0x7fba480b1a80 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba46ffd700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 0x7fba480c6c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fba3800e9b0 tx=0x7fba3800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba38009710 con 0x7fba480b1a80 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3800ee50 con 0x7fba480b1a80 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3800c9f0 con 0x7fba480b1a80 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.669+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba480c91a0 con 0x7fba480b1a80 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.670+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba481464e0 con 0x7fba480b1a80 2023-07-06T21:41:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.671+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba48146850 con 0x7fba480b1a80 2023-07-06T21:41:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.672+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba3800cb50 con 0x7fba480b1a80 2023-07-06T21:41:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.672+0000 7fba2f7fe700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba3406ace0 0x7fba3406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:40.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.673+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fba38089940 con 0x7fba480b1a80 2023-07-06T21:41:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.673+0000 7fba467fc700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba3406ace0 0x7fba3406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:40.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.673+0000 7fba467fc700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba3406ace0 0x7fba3406d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0090b0 tx=0x7fba3c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:40.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.677+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba380582f0 con 0x7fba480b1a80 2023-07-06T21:41:40.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.871+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fba480d3360 con 0x7fba480b1a80 2023-07-06T21:41:40.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.871+0000 7fba2f7fe700 1 -- 172.21.15.84:0/2896067454 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fba38057e80 con 0x7fba480b1a80 2023-07-06T21:41:40.875 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:40.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba3406ace0 msgr2=0x7fba3406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba3406ace0 0x7fba3406d190 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0090b0 tx=0x7fba3c008030 comp rx=0 tx=0).stop 2023-07-06T21:41:40.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 msgr2=0x7fba480c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:40.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 0x7fba480c6c50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fba3800e9b0 tx=0x7fba3800ecc0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 shutdown_connections 2023-07-06T21:41:40.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba3406ace0 0x7fba3406d190 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba480ceda0 0x7fba481e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba480b1a80 0x7fba480c6c50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 --2- 172.21.15.84:0/2896067454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba480b81c0 0x7fba480c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:40.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 >> 172.21.15.84:0/2896067454 conn(0x7fba4806b2b0 msgr2=0x7fba480b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:40.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 shutdown_connections 2023-07-06T21:41:40.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:40.873+0000 7fba4d3ff700 1 -- 172.21.15.84:0/2896067454 wait complete. 2023-07-06T21:41:40.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:41.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.089+0000 7fda537ff700 1 Processor -- start 2023-07-06T21:41:41.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.089+0000 7fda537ff700 1 -- start start 2023-07-06T21:41:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda537ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b81c0 0x7fda4c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda537ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda4c0b1a80 0x7fda4c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda537ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda4c0d1830 con 0x7fda4c0ceda0 2023-07-06T21:41:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda537ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda4c0cf330 con 0x7fda4c0b1a80 2023-07-06T21:41:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda537ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0d1270 con 0x7fda4c0b81c0 2023-07-06T21:41:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda50d9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda4c0b1a80 0x7fda4c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.090+0000 7fda50d9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fda4c0b1a80 0x7fda4c0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34158/0 (socket says 172.21.15.84:34158) 2023-07-06T21:41:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda51d9c700 1 --1- 172.21.15.84:0/3000383085 >> v1:172.21.15.84:6789/0 conn(0x7fda4c0ceda0 0x7fda4c15ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42848/0 (socket says 172.21.15.84:42848) 2023-07-06T21:41:41.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda50d9a700 1 -- 172.21.15.84:0/3000383085 learned_addr learned my addr 172.21.15.84:0/3000383085 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:41.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda5159b700 1 --2- 172.21.15.84:0/3000383085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b81c0 0x7fda4c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda50d9a700 1 -- 172.21.15.84:0/3000383085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b81c0 msgr2=0x7fda4c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:41.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda50d9a700 1 --2- 172.21.15.84:0/3000383085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b81c0 0x7fda4c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda50d9a700 1 -- 172.21.15.84:0/3000383085 >> v1:172.21.15.84:6789/0 conn(0x7fda4c0ceda0 legacy=0x7fda4c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda50d9a700 1 -- 172.21.15.84:0/3000383085 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0d1550 con 0x7fda4c0b1a80 2023-07-06T21:41:41.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda43fff700 1 -- 172.21.15.84:0/3000383085 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089051090 0 0) 0x7fda4c0d1830 con 0x7fda4c0ceda0 2023-07-06T21:41:41.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda50d9a700 1 --2- 172.21.15.84:0/3000383085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b1a80 0x7fda4c0cfb40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0098f0 tx=0x7fda3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df9a47469518793 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda43fff700 1 -- 172.21.15.84:0/3000383085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011830 con 0x7fda4c0b1a80 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.091+0000 7fda43fff700 1 -- 172.21.15.84:0/3000383085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c011990 con 0x7fda4c0b1a80 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda43fff700 1 -- 172.21.15.84:0/3000383085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c011d20 con 0x7fda4c0b1a80 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 -- 172.21.15.84:0/3000383085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b1a80 msgr2=0x7fda4c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 --2- 172.21.15.84:0/3000383085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b1a80 0x7fda4c0cfb40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0098f0 tx=0x7fda3c009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 -- 172.21.15.84:0/3000383085 shutdown_connections 2023-07-06T21:41:41.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 --2- 172.21.15.84:0/3000383085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b81c0 0x7fda4c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 --2- 172.21.15.84:0/3000383085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b1a80 0x7fda4c0cfb40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 -- 172.21.15.84:0/3000383085 >> 172.21.15.84:0/3000383085 conn(0x7fda4c06b2b0 msgr2=0x7fda4c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 -- 172.21.15.84:0/3000383085 shutdown_connections 2023-07-06T21:41:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.092+0000 7fda537ff700 1 -- 172.21.15.84:0/3000383085 wait complete. 2023-07-06T21:41:41.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.094+0000 7fda537ff700 1 Processor -- start 2023-07-06T21:41:41.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.094+0000 7fda537ff700 1 -- start start 2023-07-06T21:41:41.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda537ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b1a80 0x7fda4c0cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda537ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 0x7fda4c1e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda537ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda4c0ceda0 0x7fda4c148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda537ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0cc470 con 0x7fda4c0b81c0 2023-07-06T21:41:41.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda537ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0cb7f0 con 0x7fda4c0ceda0 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda537ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c0cbad0 con 0x7fda4c0b1a80 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda50d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 0x7fda4c1e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda50d9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 0x7fda4c1e1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34164/0 (socket says 172.21.15.84:34164) 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.095+0000 7fda50d9a700 1 -- 172.21.15.84:0/297946451 learned_addr learned my addr 172.21.15.84:0/297946451 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda51d9c700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda4c0ceda0 0x7fda4c148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda5159b700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b1a80 0x7fda4c0cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda50d9a700 1 -- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b1a80 msgr2=0x7fda4c0cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda50d9a700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b1a80 0x7fda4c0cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda50d9a700 1 -- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda4c0ceda0 msgr2=0x7fda4c148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda50d9a700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda4c0ceda0 0x7fda4c148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda50d9a700 1 -- 172.21.15.84:0/297946451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009550 con 0x7fda4c0b81c0 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda51d9c700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda4c0ceda0 0x7fda4c148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda5159b700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b1a80 0x7fda4c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda50d9a700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 0x7fda4c1e1730 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009c40 tx=0x7fda3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c004480 con 0x7fda4c0b81c0 2023-07-06T21:41:41.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c01f070 con 0x7fda4c0b81c0 2023-07-06T21:41:41.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c0cbdb0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0038c0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.096+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c0cb480 con 0x7fda4c0b81c0 2023-07-06T21:41:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.097+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4c0cc5b0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.099+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fda3c021030 con 0x7fda4c0b81c0 2023-07-06T21:41:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.100+0000 7fda41ffb700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda3006ace0 0x7fda3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.100+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fda3c08aab0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.100+0000 7fda5159b700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda3006ace0 0x7fda3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.100+0000 7fda5159b700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda3006ace0 0x7fda3006d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fda48009b60 tx=0x7fda48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.104+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fda4c0cc5b0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.302+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda4c0cc5b0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.303+0000 7fda41ffb700 1 -- 172.21.15.84:0/297946451 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda4c0cc5b0 con 0x7fda4c0b81c0 2023-07-06T21:41:41.306 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:41.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda3006ace0 msgr2=0x7fda3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda3006ace0 0x7fda3006d190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fda48009b60 tx=0x7fda48008040 comp rx=0 tx=0).stop 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 msgr2=0x7fda4c1e1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 0x7fda4c1e1730 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009c40 tx=0x7fda3c004750 comp rx=0 tx=0).stop 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 shutdown_connections 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fda3006ace0 0x7fda3006d190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda4c0ceda0 0x7fda4c148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda4c0b1a80 0x7fda4c0cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 --2- 172.21.15.84:0/297946451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fda4c0b81c0 0x7fda4c1e1730 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 >> 172.21.15.84:0/297946451 conn(0x7fda4c06b2b0 msgr2=0x7fda4c0b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:41.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 shutdown_connections 2023-07-06T21:41:41.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.305+0000 7fda537ff700 1 -- 172.21.15.84:0/297946451 wait complete. 2023-07-06T21:41:41.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:41.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.519+0000 7f937c9ff700 1 Processor -- start 2023-07-06T21:41:41.522 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.519+0000 7f937c9ff700 1 -- start start 2023-07-06T21:41:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.520+0000 7f937c9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93780cfd10 0x7f93780d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.520+0000 7f937c9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f93780b1a80 0x7f93780d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.520+0000 7f937c9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93780d1830 con 0x7f93780b81c0 2023-07-06T21:41:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.520+0000 7f937c9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93780cf330 con 0x7f93780cfd10 2023-07-06T21:41:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.520+0000 7f937c9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93780d1270 con 0x7f93780b1a80 2023-07-06T21:41:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93780cfd10 0x7f93780d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f93780cfd10 0x7f93780d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34166/0 (socket says 172.21.15.84:34166) 2023-07-06T21:41:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 -- 172.21.15.84:0/3472806079 learned_addr learned my addr 172.21.15.84:0/3472806079 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9376d9d700 1 --2- 172.21.15.84:0/3472806079 >> v2:172.21.15.107:3300/0 conn(0x7f93780b1a80 0x7f93780d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 -- 172.21.15.84:0/3472806079 >> v2:172.21.15.107:3300/0 conn(0x7f93780b1a80 msgr2=0x7f93780d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 --2- 172.21.15.84:0/3472806079 >> v2:172.21.15.107:3300/0 conn(0x7f93780b1a80 0x7f93780d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 -- 172.21.15.84:0/3472806079 >> v1:172.21.15.107:6789/0 conn(0x7f93780b81c0 legacy=0x7f93780cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 -- 172.21.15.84:0/3472806079 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93780d1550 con 0x7f93780cfd10 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9376d9d700 1 --2- 172.21.15.84:0/3472806079 >> v2:172.21.15.107:3300/0 conn(0x7f93780b1a80 0x7f93780d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.521+0000 7f9375d9b700 1 --2- 172.21.15.84:0/3472806079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780cfd10 0x7f93780d2380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f93780b8d30 tx=0x7f936c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daeabfd7e80a6d7b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937559a700 1 -- 172.21.15.84:0/3472806079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c00b870 con 0x7f93780cfd10 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937559a700 1 -- 172.21.15.84:0/3472806079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c00b9d0 con 0x7f93780cfd10 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937559a700 1 -- 172.21.15.84:0/3472806079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c00bd60 con 0x7f93780cfd10 2023-07-06T21:41:41.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 -- 172.21.15.84:0/3472806079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780cfd10 msgr2=0x7f93780d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 --2- 172.21.15.84:0/3472806079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780cfd10 0x7f93780d2380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f93780b8d30 tx=0x7f936c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 -- 172.21.15.84:0/3472806079 shutdown_connections 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 --2- 172.21.15.84:0/3472806079 >> v2:172.21.15.107:3300/0 conn(0x7f93780b1a80 0x7f93780d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 --2- 172.21.15.84:0/3472806079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780cfd10 0x7f93780d2380 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 -- 172.21.15.84:0/3472806079 >> 172.21.15.84:0/3472806079 conn(0x7f937806b2b0 msgr2=0x7f937806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 -- 172.21.15.84:0/3472806079 shutdown_connections 2023-07-06T21:41:41.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.522+0000 7f937c9ff700 1 -- 172.21.15.84:0/3472806079 wait complete. 2023-07-06T21:41:41.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.524+0000 7f937c9ff700 1 Processor -- start 2023-07-06T21:41:41.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.524+0000 7f937c9ff700 1 -- start start 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f937c9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93780b1a80 0x7f93780cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f937c9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 0x7f9378157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f937c9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93780cfd10 0x7f93781582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f937c9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93780ccc00 con 0x7f93780b81c0 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f937c9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9378142e20 con 0x7f93780b1a80 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f937c9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9378143100 con 0x7f93780cfd10 2023-07-06T21:41:41.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f9375d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 0x7f9378157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f9375d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 0x7f9378157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34170/0 (socket says 172.21.15.84:34170) 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.525+0000 7f9375d9b700 1 -- 172.21.15.84:0/1527898164 learned_addr learned my addr 172.21.15.84:0/1527898164 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f937659c700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93780b1a80 0x7f93780cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9376d9d700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93780cfd10 0x7f93781582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9375d9b700 1 -- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93780cfd10 msgr2=0x7f93781582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9375d9b700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93780cfd10 0x7f93781582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9375d9b700 1 -- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93780b1a80 msgr2=0x7f93780cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9375d9b700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93780b1a80 0x7f93780cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9375d9b700 1 -- 172.21.15.84:0/1527898164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c009550 con 0x7f93780b81c0 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9376d9d700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93780cfd10 0x7f93781582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9375d9b700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 0x7f9378157db0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f936c009c10 tx=0x7f936c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c004540 con 0x7f93780b81c0 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c0036a0 con 0x7f93780b81c0 2023-07-06T21:41:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c003a50 con 0x7f93780b81c0 2023-07-06T21:41:41.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93781433e0 con 0x7f93780b81c0 2023-07-06T21:41:41.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.526+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9378145cb0 con 0x7f93780b81c0 2023-07-06T21:41:41.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.527+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9378143900 con 0x7f93780b81c0 2023-07-06T21:41:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.528+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f936c021070 con 0x7f93780b81c0 2023-07-06T21:41:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.529+0000 7f9366ffd700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f935806ace0 0x7f935806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.529+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f936c08ad50 con 0x7f93780b81c0 2023-07-06T21:41:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.529+0000 7f937659c700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f935806ace0 0x7f935806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.530+0000 7f937659c700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f935806ace0 0x7f935806d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9368000f60 tx=0x7f9368009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.533+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9378143900 con 0x7f93780b81c0 2023-07-06T21:41:41.735 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.732+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9378143900 con 0x7f93780b81c0 2023-07-06T21:41:41.736 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.733+0000 7f9366ffd700 1 -- 172.21.15.84:0/1527898164 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9378143900 con 0x7f93780b81c0 2023-07-06T21:41:41.736 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:41.738 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f935806ace0 msgr2=0x7f935806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f935806ace0 0x7f935806d190 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9368000f60 tx=0x7f9368009040 comp rx=0 tx=0).stop 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 msgr2=0x7f9378157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 0x7f9378157db0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f936c009c10 tx=0x7f936c004750 comp rx=0 tx=0).stop 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 shutdown_connections 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f935806ace0 0x7f935806d190 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93780cfd10 0x7f93781582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.739 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93780b1a80 0x7f93780cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 --2- 172.21.15.84:0/1527898164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93780b81c0 0x7f9378157db0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 >> 172.21.15.84:0/1527898164 conn(0x7f937806b2b0 msgr2=0x7f93780d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:41.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 shutdown_connections 2023-07-06T21:41:41.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.735+0000 7f937c9ff700 1 -- 172.21.15.84:0/1527898164 wait complete. 2023-07-06T21:41:41.756 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.951+0000 7f01e7bff700 1 Processor -- start 2023-07-06T21:41:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.952+0000 7f01e7bff700 1 -- start start 2023-07-06T21:41:41.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.953+0000 7f01e7bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f01e00ce8c0 0x7f01e006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.953+0000 7f01e7bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e00cd880 con 0x7f01e006cc80 2023-07-06T21:41:41.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.953+0000 7f01e7bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e006e4a0 con 0x7f01e006de70 2023-07-06T21:41:41.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.953+0000 7f01e7bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00cdb00 con 0x7f01e00ce8c0 2023-07-06T21:41:41.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.953+0000 7f01e599b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f01e006cc80 0x7f01e006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42852/0 (socket says 172.21.15.84:42852) 2023-07-06T21:41:41.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e599b700 1 -- 172.21.15.84:0/107067452 learned_addr learned my addr 172.21.15.84:0/107067452 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:41.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e519a700 1 --2- 172.21.15.84:0/107067452 >> v2:172.21.15.84:3300/0 conn(0x7f01e00ce8c0 0x7f01e006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e4999700 1 -- 172.21.15.84:0/107067452 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913860736 0 0) 0x7f01e00cd880 con 0x7f01e006cc80 2023-07-06T21:41:41.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e4999700 1 -- 172.21.15.84:0/107067452 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01cc0035b0 con 0x7f01e006cc80 2023-07-06T21:41:41.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e519a700 1 -- 172.21.15.84:0/107067452 >> v1:172.21.15.107:6789/0 conn(0x7f01e006de70 legacy=0x7f01e0069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e519a700 1 -- 172.21.15.84:0/107067452 >> v1:172.21.15.84:6789/0 conn(0x7f01e006cc80 legacy=0x7f01e006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e519a700 1 -- 172.21.15.84:0/107067452 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00cdde0 con 0x7f01e00ce8c0 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.954+0000 7f01e519a700 1 --2- 172.21.15.84:0/107067452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e00ce8c0 0x7f01e006d930 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009840 tx=0x7f01d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8503a9bf52ee2231 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e4999700 1 -- 172.21.15.84:0/107067452 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0011830 con 0x7f01e00ce8c0 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e4999700 1 -- 172.21.15.84:0/107067452 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0011990 con 0x7f01e00ce8c0 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e4999700 1 -- 172.21.15.84:0/107067452 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0011d00 con 0x7f01e00ce8c0 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 -- 172.21.15.84:0/107067452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e00ce8c0 msgr2=0x7f01e006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 --2- 172.21.15.84:0/107067452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e00ce8c0 0x7f01e006d930 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009840 tx=0x7f01d0009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:41.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 -- 172.21.15.84:0/107067452 shutdown_connections 2023-07-06T21:41:41.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 --2- 172.21.15.84:0/107067452 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e00ce8c0 0x7f01e006d930 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 -- 172.21.15.84:0/107067452 >> 172.21.15.84:0/107067452 conn(0x7f01e00cf080 msgr2=0x7f01e00cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:41.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 -- 172.21.15.84:0/107067452 shutdown_connections 2023-07-06T21:41:41.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.955+0000 7f01e7bff700 1 -- 172.21.15.84:0/107067452 wait complete. 2023-07-06T21:41:41.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.957+0000 7f01e7bff700 1 Processor -- start 2023-07-06T21:41:41.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.957+0000 7f01e7bff700 1 -- start start 2023-07-06T21:41:41.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e7bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e006cc80 0x7f01e0158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e7bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 0x7f01e0156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e7bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e00ce8c0 0x7f01e009de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e7bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e0156aa0 con 0x7f01e006de70 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e7bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e015f9a0 con 0x7f01e00ce8c0 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e7bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00b2940 con 0x7f01e006cc80 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e519a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 0x7f01e0156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e519a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 0x7f01e0156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34188/0 (socket says 172.21.15.84:34188) 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e519a700 1 -- 172.21.15.84:0/115643319 learned_addr learned my addr 172.21.15.84:0/115643319 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:41.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e599b700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e006cc80 0x7f01e0158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e619c700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e00ce8c0 0x7f01e009de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.958+0000 7f01e519a700 1 -- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e006cc80 msgr2=0x7f01e0158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e519a700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e006cc80 0x7f01e0158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e519a700 1 -- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e00ce8c0 msgr2=0x7f01e009de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e519a700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e00ce8c0 0x7f01e009de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e519a700 1 -- 172.21.15.84:0/115643319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d00094a0 con 0x7f01e006de70 2023-07-06T21:41:41.965 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e619c700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e00ce8c0 0x7f01e009de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:41.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e519a700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 0x7f01e0156560 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000c00 tx=0x7f01d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d00036a0 con 0x7f01e006de70 2023-07-06T21:41:41.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d001f070 con 0x7f01e006de70 2023-07-06T21:41:41.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e00b2050 con 0x7f01e006de70 2023-07-06T21:41:41.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0003a00 con 0x7f01e006de70 2023-07-06T21:41:41.966 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.959+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e00b25d0 con 0x7f01e006de70 2023-07-06T21:41:41.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.960+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e004e1d0 con 0x7f01e006de70 2023-07-06T21:41:41.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.961+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f01d0003b60 con 0x7f01e006de70 2023-07-06T21:41:41.967 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.962+0000 7f01d67fc700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f01c406ace0 0x7f01c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:41.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.962+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f01d0021070 con 0x7f01e006de70 2023-07-06T21:41:41.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.963+0000 7f01e599b700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f01c406ace0 0x7f01c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:41.968 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.963+0000 7f01e599b700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f01c406ace0 0x7f01c406d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0011e0 tx=0x7f01dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:41.969 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:41.966+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f01d0055df0 con 0x7f01e006de70 2023-07-06T21:41:42.168 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.165+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f01e0071a00 con 0x7f01e006de70 2023-07-06T21:41:42.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.166+0000 7f01d67fc700 1 -- 172.21.15.84:0/115643319 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7f01d00150a0 con 0x7f01e006de70 2023-07-06T21:41:42.169 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:42.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f01c406ace0 msgr2=0x7f01c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f01c406ace0 0x7f01c406d190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0011e0 tx=0x7f01dc009040 comp rx=0 tx=0).stop 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 msgr2=0x7f01e0156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 0x7f01e0156560 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f01d0000c00 tx=0x7f01d0004750 comp rx=0 tx=0).stop 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 shutdown_connections 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f01c406ace0 0x7f01c406d190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01e00ce8c0 0x7f01e009de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01e006cc80 0x7f01e0158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 --2- 172.21.15.84:0/115643319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01e006de70 0x7f01e0156560 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 >> 172.21.15.84:0/115643319 conn(0x7f01e00cf080 msgr2=0x7f01e009ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:42.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 shutdown_connections 2023-07-06T21:41:42.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.168+0000 7f01e7bff700 1 -- 172.21.15.84:0/115643319 wait complete. 2023-07-06T21:41:42.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:41:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.381+0000 7f9d9b5ff700 1 Processor -- start 2023-07-06T21:41:42.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.381+0000 7f9d9b5ff700 1 -- start start 2023-07-06T21:41:42.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d9b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d94158830 0x7f9d940c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:42.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d9b5ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9d9406a5a0 0x7f9d940c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:42.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d9b5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d940c3f80 con 0x7f9d940c5280 2023-07-06T21:41:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d9b5ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d940c4260 con 0x7f9d9406a5a0 2023-07-06T21:41:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d9b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c4540 con 0x7f9d94158830 2023-07-06T21:41:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d98b9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9d940c5280 0x7f9d94069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:42856/0 (socket says 172.21.15.84:42856) 2023-07-06T21:41:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d98b9a700 1 -- 172.21.15.84:0/2719374653 learned_addr learned my addr 172.21.15.84:0/2719374653 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d99b9c700 1 --2- 172.21.15.84:0/2719374653 >> v2:172.21.15.84:3300/0 conn(0x7f9d9406a5a0 0x7f9d940c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:42.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.383+0000 7f9d9939b700 1 --2- 172.21.15.84:0/2719374653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d94158830 0x7f9d940c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d99b9c700 1 -- 172.21.15.84:0/2719374653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d94158830 msgr2=0x7f9d940c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d99b9c700 1 --2- 172.21.15.84:0/2719374653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d94158830 0x7f9d940c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d99b9c700 1 -- 172.21.15.84:0/2719374653 >> v1:172.21.15.84:6789/0 conn(0x7f9d940c5280 legacy=0x7f9d94069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d99b9c700 1 -- 172.21.15.84:0/2719374653 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d940c4820 con 0x7f9d9406a5a0 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d8bfff700 1 -- 172.21.15.84:0/2719374653 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529230558 0 0) 0x7f9d940c3f80 con 0x7f9d940c5280 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d99b9c700 1 --2- 172.21.15.84:0/2719374653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d940c92c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d90 tx=0x7f9d8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c3c9b85ad2c558 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:42.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d8bfff700 1 -- 172.21.15.84:0/2719374653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c00e070 con 0x7f9d9406a5a0 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d8bfff700 1 -- 172.21.15.84:0/2719374653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c00bb20 con 0x7f9d9406a5a0 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.384+0000 7f9d8bfff700 1 -- 172.21.15.84:0/2719374653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c012430 con 0x7f9d9406a5a0 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/2719374653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 msgr2=0x7f9d940c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/2719374653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d940c92c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d90 tx=0x7f9d8c00b330 comp rx=0 tx=0).stop 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/2719374653 shutdown_connections 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/2719374653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d940c92c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/2719374653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d94158830 0x7f9d940c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/2719374653 >> 172.21.15.84:0/2719374653 conn(0x7f9d940cf3d0 msgr2=0x7f9d940cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:42.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/2719374653 shutdown_connections 2023-07-06T21:41:42.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.385+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/2719374653 wait complete. 2023-07-06T21:41:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.387+0000 7f9d9b5ff700 1 Processor -- start 2023-07-06T21:41:42.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.387+0000 7f9d9b5ff700 1 -- start start 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9b5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d941f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d940c5280 0x7f9d94156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9b5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d94158830 0x7f9d940ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9b5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941f27e0 con 0x7f9d9406a5a0 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9b5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d941f2ac0 con 0x7f9d94158830 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d940c9e30 con 0x7f9d940c5280 2023-07-06T21:41:42.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9939b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d941f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9939b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d941f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34198/0 (socket says 172.21.15.84:34198) 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.388+0000 7f9d9939b700 1 -- 172.21.15.84:0/1154085925 learned_addr learned my addr 172.21.15.84:0/1154085925 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d98b9a700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d940c5280 0x7f9d94156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d99b9c700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d94158830 0x7f9d940ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9939b700 1 -- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d940c5280 msgr2=0x7f9d94156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9939b700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d940c5280 0x7f9d94156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9939b700 1 -- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d94158830 msgr2=0x7f9d940ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9939b700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d94158830 0x7f9d940ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9939b700 1 -- 172.21.15.84:0/1154085925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c009a70 con 0x7f9d9406a5a0 2023-07-06T21:41:42.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d99b9c700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d94158830 0x7f9d940ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:42.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d98b9a700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d940c5280 0x7f9d94156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:42.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9939b700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d941f1ed0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000e870 tx=0x7f9d9000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:42.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90009590 con 0x7f9d9406a5a0 2023-07-06T21:41:42.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000ee70 con 0x7f9d9406a5a0 2023-07-06T21:41:42.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d940ca110 con 0x7f9d9406a5a0 2023-07-06T21:41:42.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000ca50 con 0x7f9d9406a5a0 2023-07-06T21:41:42.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.389+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d94157390 con 0x7f9d9406a5a0 2023-07-06T21:41:42.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.390+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d94152bd0 con 0x7f9d9406a5a0 2023-07-06T21:41:42.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.391+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d9000cbb0 con 0x7f9d9406a5a0 2023-07-06T21:41:42.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.392+0000 7f9d89ffb700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d7806ace0 0x7f9d7806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:42.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.392+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9d900895e0 con 0x7f9d9406a5a0 2023-07-06T21:41:42.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.393+0000 7f9d98b9a700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d7806ace0 0x7f9d7806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:42.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.393+0000 7f9d98b9a700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d7806ace0 0x7f9d7806d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9d94157960 tx=0x7f9d84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:42.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.397+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d90057f90 con 0x7f9d9406a5a0 2023-07-06T21:41:42.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.606+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9d941f2c80 con 0x7f9d9406a5a0 2023-07-06T21:41:42.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.606+0000 7f9d89ffb700 1 -- 172.21.15.84:0/1154085925 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9d9000ce60 con 0x7f9d9406a5a0 2023-07-06T21:41:42.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d7806ace0 msgr2=0x7f9d7806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d7806ace0 0x7f9d7806d190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9d94157960 tx=0x7f9d84009040 comp rx=0 tx=0).stop 2023-07-06T21:41:42.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 msgr2=0x7f9d941f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:42.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d941f1ed0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000e870 tx=0x7f9d9000ec30 comp rx=0 tx=0).stop 2023-07-06T21:41:42.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 shutdown_connections 2023-07-06T21:41:42.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d7806ace0 0x7f9d7806d190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d94158830 0x7f9d940ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d9406a5a0 0x7f9d941f1ed0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 --2- 172.21.15.84:0/1154085925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d940c5280 0x7f9d94156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:42.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 >> 172.21.15.84:0/1154085925 conn(0x7f9d940cf3d0 msgr2=0x7f9d94155e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:42.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 shutdown_connections 2023-07-06T21:41:42.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:42.609+0000 7f9d9b5ff700 1 -- 172.21.15.84:0/1154085925 wait complete. 2023-07-06T21:41:42.616 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:41:42.630 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:41:42.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:42.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:41:42.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[149998]: fuse finished with error 0 and tester_r 0 2023-07-06T21:41:42.700 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:41:48.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:41:48.150 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:41:48.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:41:48.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:48.803 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:41:48.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-07-06T21:41:49.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.055+0000 7f8fd17ff700 1 Processor -- start 2023-07-06T21:41:49.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.055+0000 7f8fd17ff700 1 -- start start 2023-07-06T21:41:49.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fd17ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8fcc0c8d50 0x7f8fcc0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fd17ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc15cd30 con 0x7f8fcc0703a0 2023-07-06T21:41:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fd17ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc15cf80 con 0x7f8fcc159420 2023-07-06T21:41:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fd17ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0ca800 con 0x7f8fcc0c8d50 2023-07-06T21:41:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fca7fc700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8fcc0c8d50 0x7f8fcc0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fcaffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8fcc0703a0 0x7f8fcc070770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58614/0 (socket says 172.21.15.84:58614) 2023-07-06T21:41:49.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.056+0000 7f8fcaffd700 1 -- 172.21.15.84:0/1116930700 learned_addr learned my addr 172.21.15.84:0/1116930700 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fca7fc700 1 -- 172.21.15.84:0/1116930700 >> v1:172.21.15.107:6789/0 conn(0x7f8fcc159420 legacy=0x7f8fcc159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fca7fc700 1 -- 172.21.15.84:0/1116930700 >> v1:172.21.15.84:6789/0 conn(0x7f8fcc0703a0 legacy=0x7f8fcc070770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fca7fc700 1 -- 172.21.15.84:0/1116930700 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc15d260 con 0x7f8fcc0c8d50 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fc9ffb700 1 -- 172.21.15.84:0/1116930700 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482932490 0 0) 0x7f8fcc15cd30 con 0x7f8fcc0703a0 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fca7fc700 1 --2- 172.21.15.84:0/1116930700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c91a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009840 tx=0x7f8fc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908bafd1c35ba884 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fc9ffb700 1 -- 172.21.15.84:0/1116930700 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0011830 con 0x7f8fcc0c8d50 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fc9ffb700 1 -- 172.21.15.84:0/1116930700 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc0011990 con 0x7f8fcc0c8d50 2023-07-06T21:41:49.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.057+0000 7f8fc9ffb700 1 -- 172.21.15.84:0/1116930700 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0011d20 con 0x7f8fcc0c8d50 2023-07-06T21:41:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 -- 172.21.15.84:0/1116930700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0c8d50 msgr2=0x7f8fcc0c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/1116930700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c91a0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009840 tx=0x7f8fc0009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 -- 172.21.15.84:0/1116930700 shutdown_connections 2023-07-06T21:41:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/1116930700 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c91a0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 -- 172.21.15.84:0/1116930700 >> 172.21.15.84:0/1116930700 conn(0x7f8fcc0cbe10 msgr2=0x7f8fcc0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:49.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 -- 172.21.15.84:0/1116930700 shutdown_connections 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.058+0000 7f8fd17ff700 1 -- 172.21.15.84:0/1116930700 wait complete. 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.060+0000 7f8fd17ff700 1 Processor -- start 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.060+0000 7f8fd17ff700 1 -- start start 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fd17ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 0x7f8fcc152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fd17ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fd17ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc159420 0x7f8fcc14f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fd17ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0c79d0 con 0x7f8fcc0703a0 2023-07-06T21:41:49.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fd17ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc14fca0 con 0x7f8fcc159420 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fd17ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0c7c70 con 0x7f8fcc0c8d50 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 0x7f8fcc152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 0x7f8fcc152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49328/0 (socket says 172.21.15.84:49328) 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 -- 172.21.15.84:0/330254523 learned_addr learned my addr 172.21.15.84:0/330254523 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcb7fe700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc159420 0x7f8fcc14f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fca7fc700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:49.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 -- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fcc0c8d50 msgr2=0x7f8fcc0c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 -- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc159420 msgr2=0x7f8fcc14f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.061+0000 7f8fcaffd700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc159420 0x7f8fcc14f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fcaffd700 1 -- 172.21.15.84:0/330254523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00094a0 con 0x7f8fcc0703a0 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fcb7fe700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc159420 0x7f8fcc14f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fca7fc700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fcaffd700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 0x7f8fcc152610 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c520 tx=0x7f8fbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:49.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc004c80 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc004e20 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc007310 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc0b1530 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.062+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0b1af0 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.063+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc04e1d0 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.064+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8fbc007470 con 0x7f8fcc0703a0 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.065+0000 7f8fb37fe700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8fb806ace0 0x7f8fb806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:49.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.065+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8fbc089300 con 0x7f8fcc0703a0 2023-07-06T21:41:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.065+0000 7f8fca7fc700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8fb806ace0 0x7f8fb806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.066+0000 7f8fca7fc700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8fb806ace0 0x7f8fb806d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc152bd0 tx=0x7f8fc001c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:49.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.070+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8fbc054210 con 0x7f8fcc0703a0 2023-07-06T21:41:49.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.341+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f8fcc0b4680 con 0x7f8fcc0703a0 2023-07-06T21:41:49.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.346+0000 7f8fb37fe700 1 -- 172.21.15.84:0/330254523 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7f8fbc057830 con 0x7f8fcc0703a0 2023-07-06T21:41:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8fb806ace0 msgr2=0x7f8fb806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8fb806ace0 0x7f8fb806d190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc152bd0 tx=0x7f8fc001c040 comp rx=0 tx=0).stop 2023-07-06T21:41:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 msgr2=0x7f8fcc152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 0x7f8fcc152610 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00c520 tx=0x7f8fbc00c8e0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 shutdown_connections 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8fb806ace0 0x7f8fb806d190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc159420 0x7f8fcc14f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8fcc0703a0 0x7f8fcc152610 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 --2- 172.21.15.84:0/330254523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fcc0c8d50 0x7f8fcc0c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 >> 172.21.15.84:0/330254523 conn(0x7f8fcc0cbe10 msgr2=0x7f8fcc06cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:49.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 shutdown_connections 2023-07-06T21:41:49.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:49.348+0000 7f8fd17ff700 1 -- 172.21.15.84:0/330254523 wait complete. 2023-07-06T21:41:49.353 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:41:49.369 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T21:41:49.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:41:49.469 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:41:49.573 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T21:41:49.573 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:49.574 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:49.616 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:41:49.617 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:41:49.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:49.617 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:41:49.700 INFO:teuthology.orchestra.run.smithi084.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-07-06T21:41:49.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:41:49.701 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:41:49.702 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:49.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:41:49.746 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:41:49.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:49.747 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T21:41:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-07-06T21:41:49.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:41:49.829 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:41:49.829 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:49.871 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:41:49.872 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:41:49.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:49.873 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T21:41:49.954 INFO:teuthology.orchestra.run.smithi084.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T21:41:49.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:41:49.955 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:41:49.956 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:41:49.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:41:50.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:41:50.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:41:50.000 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T21:41:50.081 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T21:41:50.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:41:50.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:41:50.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.274+0000 7f8d6f3ff700 1 Processor -- start 2023-07-06T21:41:50.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.274+0000 7f8d6f3ff700 1 -- start start 2023-07-06T21:41:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6f3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d6806a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6f3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d680c92c0 0x7f8d680c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6f3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6806a820 con 0x7f8d68158830 2023-07-06T21:41:50.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6f3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d680c3f80 con 0x7f8d680c92c0 2023-07-06T21:41:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6f3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c4260 con 0x7f8d68069e90 2023-07-06T21:41:50.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6d99c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d680c92c0 0x7f8d680c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6d99c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8d680c92c0 0x7f8d680c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49338/0 (socket says 172.21.15.84:49338) 2023-07-06T21:41:50.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.275+0000 7f8d6d99c700 1 -- 172.21.15.84:0/2553384297 learned_addr learned my addr 172.21.15.84:0/2553384297 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:50.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d6c99a700 1 --2- 172.21.15.84:0/2553384297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d6806a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d6d99c700 1 -- 172.21.15.84:0/2553384297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 msgr2=0x7f8d6806a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:50.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d6d99c700 1 --2- 172.21.15.84:0/2553384297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d6806a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d6d99c700 1 -- 172.21.15.84:0/2553384297 >> v1:172.21.15.107:6789/0 conn(0x7f8d68158830 legacy=0x7f8d680c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d6d99c700 1 -- 172.21.15.84:0/2553384297 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680c4540 con 0x7f8d680c92c0 2023-07-06T21:41:50.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d5ffff700 1 -- 172.21.15.84:0/2553384297 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545181799 0 0) 0x7f8d64001110 con 0x7f8d68158830 2023-07-06T21:41:50.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d6d99c700 1 --2- 172.21.15.84:0/2553384297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680c9720 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000d150 tx=0x7f8d6000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1065ab4ac43fc92 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d5ffff700 1 -- 172.21.15.84:0/2553384297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60010070 con 0x7f8d680c92c0 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d5ffff700 1 -- 172.21.15.84:0/2553384297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6000dd10 con 0x7f8d680c92c0 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.276+0000 7f8d5ffff700 1 -- 172.21.15.84:0/2553384297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6000b310 con 0x7f8d680c92c0 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2553384297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 msgr2=0x7f8d680c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2553384297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680c9720 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8d6000d150 tx=0x7f8d6000d460 comp rx=0 tx=0).stop 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2553384297 shutdown_connections 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2553384297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680c9720 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2553384297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d6806a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2553384297 >> 172.21.15.84:0/2553384297 conn(0x7f8d680cf3d0 msgr2=0x7f8d680cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:50.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2553384297 shutdown_connections 2023-07-06T21:41:50.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.277+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2553384297 wait complete. 2023-07-06T21:41:50.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.279+0000 7f8d6f3ff700 1 Processor -- start 2023-07-06T21:41:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.279+0000 7f8d6f3ff700 1 -- start start 2023-07-06T21:41:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6f3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d680ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6f3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6f3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d68158830 0x7f8d680c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6f3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6814d5c0 con 0x7f8d680c92c0 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6f3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6814cf20 con 0x7f8d68158830 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6f3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c7f10 con 0x7f8d68069e90 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680ae840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49352/0 (socket says 172.21.15.84:49352) 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 -- 172.21.15.84:0/2385349399 learned_addr learned my addr 172.21.15.84:0/2385349399 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6d99c700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d68158830 0x7f8d680c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6d19b700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d680ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 -- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 msgr2=0x7f8d680ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d680ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 -- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d68158830 msgr2=0x7f8d680c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d68158830 0x7f8d680c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6c99a700 1 -- 172.21.15.84:0/2385349399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6000bd40 con 0x7f8d680c92c0 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.280+0000 7f8d6d99c700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d68158830 0x7f8d680c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:50.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.281+0000 7f8d6c99a700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680ae840 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800e870 tx=0x7f8d5800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.281+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58009590 con 0x7f8d680c92c0 2023-07-06T21:41:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.281+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58013680 con 0x7f8d680c92c0 2023-07-06T21:41:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.281+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d580139f0 con 0x7f8d680c92c0 2023-07-06T21:41:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.281+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6814d1e0 con 0x7f8d680c92c0 2023-07-06T21:41:50.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.281+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6814a830 con 0x7f8d680c92c0 2023-07-06T21:41:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.282+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6814d700 con 0x7f8d680c92c0 2023-07-06T21:41:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.283+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8d58013b50 con 0x7f8d680c92c0 2023-07-06T21:41:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.284+0000 7f8d5dffb700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d4c06ace0 0x7f8d4c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.284+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f8d58089490 con 0x7f8d680c92c0 2023-07-06T21:41:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.284+0000 7f8d6d19b700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d4c06ace0 0x7f8d4c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.285+0000 7f8d6d19b700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d4c06ace0 0x7f8d4c06d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000f60 tx=0x7f8d64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:50.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.289+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8d58057e30 con 0x7f8d680c92c0 2023-07-06T21:41:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.528+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d680c39b0 con 0x7f8d680c92c0 2023-07-06T21:41:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.529+0000 7f8d5dffb700 1 -- 172.21.15.84:0/2385349399 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f8d580579c0 con 0x7f8d680c92c0 2023-07-06T21:41:50.531 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:50.531 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:41:04.476261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:41:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d4c06ace0 msgr2=0x7f8d4c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d4c06ace0 0x7f8d4c06d190 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8d64000f60 tx=0x7f8d64009040 comp rx=0 tx=0).stop 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 msgr2=0x7f8d680ae840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680ae840 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800e870 tx=0x7f8d5800eb80 comp rx=0 tx=0).stop 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 shutdown_connections 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8d4c06ace0 0x7f8d4c06d190 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d68158830 0x7f8d680c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d68069e90 0x7f8d680ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 --2- 172.21.15.84:0/2385349399 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d680c92c0 0x7f8d680ae840 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.531+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 >> 172.21.15.84:0/2385349399 conn(0x7f8d680cf3d0 msgr2=0x7f8d681575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.532+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 shutdown_connections 2023-07-06T21:41:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.532+0000 7f8d6f3ff700 1 -- 172.21.15.84:0/2385349399 wait complete. 2023-07-06T21:41:50.535 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-07-06T21:41:50.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:50.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.743+0000 7f56e31ff700 1 Processor -- start 2023-07-06T21:41:50.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.743+0000 7f56e31ff700 1 -- start start 2023-07-06T21:41:50.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56e31ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56dc0d3710 0x7f56dc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56e31ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0cd910 con 0x7f56dc0cfa90 2023-07-06T21:41:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56e31ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56dc0d0a80 con 0x7f56dc0b83c0 2023-07-06T21:41:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56e31ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56dc0cff10 con 0x7f56dc0d3710 2023-07-06T21:41:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56e179c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f56dc0cfa90 0x7f56dc0d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58624/0 (socket says 172.21.15.84:58624) 2023-07-06T21:41:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56dbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f56dc0d3710 0x7f56dc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.745+0000 7f56e179c700 1 -- 172.21.15.84:0/1510554777 learned_addr learned my addr 172.21.15.84:0/1510554777 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:50.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56dbfff700 1 -- 172.21.15.84:0/1510554777 >> v1:172.21.15.107:6789/0 conn(0x7f56dc0b83c0 legacy=0x7f56dc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56dbfff700 1 -- 172.21.15.84:0/1510554777 >> v1:172.21.15.84:6789/0 conn(0x7f56dc0cfa90 legacy=0x7f56dc0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56db7fe700 1 -- 172.21.15.84:0/1510554777 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136774893 0 0) 0x7f56dc0d0a80 con 0x7f56dc0b83c0 2023-07-06T21:41:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56dbfff700 1 -- 172.21.15.84:0/1510554777 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc0b8790 con 0x7f56dc0d3710 2023-07-06T21:41:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56dbfff700 1 --2- 172.21.15.84:0/1510554777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0d3710 0x7f56dc0cf550 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009840 tx=0x7f56c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbff214265c772c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56db7fe700 1 -- 172.21.15.84:0/1510554777 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8004230 con 0x7f56dc0d3710 2023-07-06T21:41:50.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56db7fe700 1 -- 172.21.15.84:0/1510554777 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c8004390 con 0x7f56dc0d3710 2023-07-06T21:41:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.746+0000 7f56db7fe700 1 -- 172.21.15.84:0/1510554777 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80036a0 con 0x7f56dc0d3710 2023-07-06T21:41:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 -- 172.21.15.84:0/1510554777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0d3710 msgr2=0x7f56dc0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 --2- 172.21.15.84:0/1510554777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0d3710 0x7f56dc0cf550 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f56c8009840 tx=0x7f56c8009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:50.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 -- 172.21.15.84:0/1510554777 shutdown_connections 2023-07-06T21:41:50.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 --2- 172.21.15.84:0/1510554777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0d3710 0x7f56dc0cf550 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 -- 172.21.15.84:0/1510554777 >> 172.21.15.84:0/1510554777 conn(0x7f56dc06b350 msgr2=0x7f56dc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:50.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 -- 172.21.15.84:0/1510554777 shutdown_connections 2023-07-06T21:41:50.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.747+0000 7f56e31ff700 1 -- 172.21.15.84:0/1510554777 wait complete. 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.749+0000 7f56e31ff700 1 Processor -- start 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.749+0000 7f56e31ff700 1 -- start start 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e31ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 0x7f56dc0c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e31ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56dc0cfa90 0x7f56dc0c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e31ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56dc0d3710 0x7f56dc0c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e31ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc0c9430 con 0x7f56dc0b83c0 2023-07-06T21:41:50.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e31ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc149ee0 con 0x7f56dc0cfa90 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e31ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56dc147000 con 0x7f56dc0d3710 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 0x7f56dc0c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 0x7f56dc0c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49370/0 (socket says 172.21.15.84:49370) 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 -- 172.21.15.84:0/885361017 learned_addr learned my addr 172.21.15.84:0/885361017 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 -- 172.21.15.84:0/885361017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56dc0d3710 msgr2=0x7f56dc0c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56dc0d3710 0x7f56dc0c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 -- 172.21.15.84:0/885361017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56dc0cfa90 msgr2=0x7f56dc0c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56dc0cfa90 0x7f56dc0c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.750+0000 7f56e0f9b700 1 -- 172.21.15.84:0/885361017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80094a0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.751+0000 7f56e0f9b700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 0x7f56dc0c5ac0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f56d000c450 tx=0x7f56d000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.751+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00054e0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.751+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0003680 con 0x7f56dc0b83c0 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.751+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56dc1472e0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.751+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0003a30 con 0x7f56dc0b83c0 2023-07-06T21:41:50.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.751+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56dc1478a0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.752+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56dc0d1390 con 0x7f56dc0b83c0 2023-07-06T21:41:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.753+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f56d0003b90 con 0x7f56dc0b83c0 2023-07-06T21:41:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.754+0000 7f56d97fa700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f56c006ad10 0x7f56c006d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.754+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f56d0088be0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.754+0000 7f56dbfff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f56c006ad10 0x7f56c006d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:50.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.755+0000 7f56dbfff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f56c006ad10 0x7f56c006d1c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0d3b20 tx=0x7f56c8010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:50.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.758+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f56d00539d0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.956+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56dc0d10b0 con 0x7f56dc0b83c0 2023-07-06T21:41:50.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.957+0000 7f56d97fa700 1 -- 172.21.15.84:0/885361017 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7f56d0017020 con 0x7f56dc0b83c0 2023-07-06T21:41:50.960 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:50.960 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f56c006ad10 msgr2=0x7f56c006d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f56c006ad10 0x7f56c006d1c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0d3b20 tx=0x7f56c8010000 comp rx=0 tx=0).stop 2023-07-06T21:41:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 msgr2=0x7f56dc0c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 0x7f56dc0c5ac0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f56d000c450 tx=0x7f56d000c810 comp rx=0 tx=0).stop 2023-07-06T21:41:50.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 shutdown_connections 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f56c006ad10 0x7f56c006d1c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.959+0000 7f56e31ff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56dc0d3710 0x7f56dc0c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.960+0000 7f56e31ff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f56dc0b83c0 0x7f56dc0c5ac0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.960+0000 7f56e31ff700 1 --2- 172.21.15.84:0/885361017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56dc0cfa90 0x7f56dc0c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.960+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 >> 172.21.15.84:0/885361017 conn(0x7f56dc06b350 msgr2=0x7f56dc0d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.960+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 shutdown_connections 2023-07-06T21:41:50.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:50.960+0000 7f56e31ff700 1 -- 172.21.15.84:0/885361017 wait complete. 2023-07-06T21:41:50.981 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:40:51.215137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:40:52.645406+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'data_new', 'create_time': '2023-07-06T21:41:03.311313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:41:50.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:41:51.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.176+0000 7f61867ff700 1 Processor -- start 2023-07-06T21:41:51.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.176+0000 7f61867ff700 1 -- start start 2023-07-06T21:41:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.177+0000 7f61867ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61800b81c0 0x7f61800cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.177+0000 7f61867ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61800d1830 con 0x7f61800b1a80 2023-07-06T21:41:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.177+0000 7f61867ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6180160d40 con 0x7f61800ceda0 2023-07-06T21:41:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.177+0000 7f61867ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61800cf330 con 0x7f61800b81c0 2023-07-06T21:41:51.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.177+0000 7f617f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f61800b1a80 0x7f618015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58626/0 (socket says 172.21.15.84:58626) 2023-07-06T21:41:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617ffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f61800b81c0 0x7f61800cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617f7fe700 1 -- 172.21.15.84:0/1649956745 learned_addr learned my addr 172.21.15.84:0/1649956745 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617effd700 1 -- 172.21.15.84:0/1649956745 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035988333 0 0) 0x7f61800d1830 con 0x7f61800b1a80 2023-07-06T21:41:51.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617ffff700 1 -- 172.21.15.84:0/1649956745 >> v1:172.21.15.107:6789/0 conn(0x7f61800ceda0 legacy=0x7f6180160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617ffff700 1 -- 172.21.15.84:0/1649956745 >> v1:172.21.15.84:6789/0 conn(0x7f61800b1a80 legacy=0x7f618015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617ffff700 1 -- 172.21.15.84:0/1649956745 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61800d1270 con 0x7f61800b81c0 2023-07-06T21:41:51.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617ffff700 1 --2- 172.21.15.84:0/1649956745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b81c0 0x7f61800cf600 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6170009840 tx=0x7f6170009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c75a4ae2ddce7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:51.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617effd700 1 -- 172.21.15.84:0/1649956745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170011830 con 0x7f61800b81c0 2023-07-06T21:41:51.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617effd700 1 -- 172.21.15.84:0/1649956745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6170011990 con 0x7f61800b81c0 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.178+0000 7f617effd700 1 -- 172.21.15.84:0/1649956745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6170011d20 con 0x7f61800b81c0 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 -- 172.21.15.84:0/1649956745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b81c0 msgr2=0x7f61800cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 --2- 172.21.15.84:0/1649956745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b81c0 0x7f61800cf600 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6170009840 tx=0x7f6170009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 -- 172.21.15.84:0/1649956745 shutdown_connections 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 --2- 172.21.15.84:0/1649956745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b81c0 0x7f61800cf600 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 -- 172.21.15.84:0/1649956745 >> 172.21.15.84:0/1649956745 conn(0x7f618006b2b0 msgr2=0x7f618006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:51.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 -- 172.21.15.84:0/1649956745 shutdown_connections 2023-07-06T21:41:51.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.179+0000 7f61867ff700 1 -- 172.21.15.84:0/1649956745 wait complete. 2023-07-06T21:41:51.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.181+0000 7f61867ff700 1 Processor -- start 2023-07-06T21:41:51.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.181+0000 7f61867ff700 1 -- start start 2023-07-06T21:41:51.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f61867ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 0x7f61801f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f61867ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61800b81c0 0x7f61800b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f61867ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61800ceda0 0x7f61800b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f61867ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6180154b70 con 0x7f61800b1a80 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f61867ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61800cbb70 con 0x7f61800ceda0 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f61867ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61800cca90 con 0x7f61800b81c0 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 0x7f61801f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 0x7f61801f4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49390/0 (socket says 172.21.15.84:49390) 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 -- 172.21.15.84:0/2963444430 learned_addr learned my addr 172.21.15.84:0/2963444430 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617f7fe700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61800b81c0 0x7f61800b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f6184d9c700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61800ceda0 0x7f61800b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 -- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61800b81c0 msgr2=0x7f61800b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61800b81c0 0x7f61800b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 -- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61800ceda0 msgr2=0x7f61800b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.182+0000 7f617ffff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61800ceda0 0x7f61800b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f617ffff700 1 -- 172.21.15.84:0/2963444430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700094a0 con 0x7f61800b1a80 2023-07-06T21:41:51.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f617ffff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 0x7f61801f4b40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f618006bed0 tx=0x7f6170004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:51.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617001a8c0 con 0x7f61800b1a80 2023-07-06T21:41:51.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617001aa20 con 0x7f61800b1a80 2023-07-06T21:41:51.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61800cadb0 con 0x7f61800b1a80 2023-07-06T21:41:51.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617001ad90 con 0x7f61800b1a80 2023-07-06T21:41:51.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.183+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61800cb370 con 0x7f61800b1a80 2023-07-06T21:41:51.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.184+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618004e1d0 con 0x7f61800b1a80 2023-07-06T21:41:51.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.185+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6170022070 con 0x7f61800b1a80 2023-07-06T21:41:51.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.186+0000 7f617cff9700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f616c06ace0 0x7f616c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.186+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f617008a890 con 0x7f61800b1a80 2023-07-06T21:41:51.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.186+0000 7f617f7fe700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f616c06ace0 0x7f616c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.187+0000 7f617f7fe700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f616c06ace0 0x7f616c06d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6174007fd0 tx=0x7f6174007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:51.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.190+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6170055970 con 0x7f61800b1a80 2023-07-06T21:41:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.383+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61800b4f30 con 0x7f61800b1a80 2023-07-06T21:41:51.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.384+0000 7f617cff9700 1 -- 172.21.15.84:0/2963444430 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6170015020 con 0x7f61800b1a80 2023-07-06T21:41:51.387 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:41:51.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f616c06ace0 msgr2=0x7f616c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f616c06ace0 0x7f616c06d190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6174007fd0 tx=0x7f6174007c20 comp rx=0 tx=0).stop 2023-07-06T21:41:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 msgr2=0x7f61801f4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 0x7f61801f4b40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f618006bed0 tx=0x7f6170004750 comp rx=0 tx=0).stop 2023-07-06T21:41:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 shutdown_connections 2023-07-06T21:41:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f616c06ace0 0x7f616c06d190 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61800ceda0 0x7f61800b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61800b1a80 0x7f61801f4b40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 --2- 172.21.15.84:0/2963444430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61800b81c0 0x7f61800b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 >> 172.21.15.84:0/2963444430 conn(0x7f618006b2b0 msgr2=0x7f618006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 shutdown_connections 2023-07-06T21:41:51.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.386+0000 7f61867ff700 1 -- 172.21.15.84:0/2963444430 wait complete. 2023-07-06T21:41:51.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:41:51.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.600+0000 7f63a79ff700 1 Processor -- start 2023-07-06T21:41:51.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.600+0000 7f63a79ff700 1 -- start start 2023-07-06T21:41:51.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.601+0000 7f63a79ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63a00cfd10 0x7f63a00d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.601+0000 7f63a79ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b1a80 0x7f63a00d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.601+0000 7f63a79ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a00d1830 con 0x7f63a00b81c0 2023-07-06T21:41:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.601+0000 7f63a79ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a00cf330 con 0x7f63a00cfd10 2023-07-06T21:41:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.601+0000 7f63a79ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00d1270 con 0x7f63a00b1a80 2023-07-06T21:41:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.601+0000 7f63a4f9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63a00cfd10 0x7f63a00d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f63a00cfd10 0x7f63a00d2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49398/0 (socket says 172.21.15.84:49398) 2023-07-06T21:41:51.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 -- 172.21.15.84:0/512215167 learned_addr learned my addr 172.21.15.84:0/512215167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:51.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 -- 172.21.15.84:0/512215167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b1a80 msgr2=0x7f63a00d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:51.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 --2- 172.21.15.84:0/512215167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b1a80 0x7f63a00d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 -- 172.21.15.84:0/512215167 >> v1:172.21.15.107:6789/0 conn(0x7f63a00b81c0 legacy=0x7f63a00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:51.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 -- 172.21.15.84:0/512215167 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00d1550 con 0x7f63a00cfd10 2023-07-06T21:41:51.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f63a4f9a700 1 --2- 172.21.15.84:0/512215167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a00d2380 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f63900098f0 tx=0x7f6390009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c4a344ac65f373 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:51.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f6397fff700 1 -- 172.21.15.84:0/512215167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390004230 con 0x7f63a00cfd10 2023-07-06T21:41:51.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.602+0000 7f6397fff700 1 -- 172.21.15.84:0/512215167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390004390 con 0x7f63a00cfd10 2023-07-06T21:41:51.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f6397fff700 1 -- 172.21.15.84:0/512215167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900036a0 con 0x7f63a00cfd10 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 -- 172.21.15.84:0/512215167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 msgr2=0x7f63a00d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 --2- 172.21.15.84:0/512215167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a00d2380 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f63900098f0 tx=0x7f6390009c00 comp rx=0 tx=0).stop 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 -- 172.21.15.84:0/512215167 shutdown_connections 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 --2- 172.21.15.84:0/512215167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b1a80 0x7f63a00d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 --2- 172.21.15.84:0/512215167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a00d2380 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 -- 172.21.15.84:0/512215167 >> 172.21.15.84:0/512215167 conn(0x7f63a006b2b0 msgr2=0x7f63a006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:51.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 -- 172.21.15.84:0/512215167 shutdown_connections 2023-07-06T21:41:51.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.603+0000 7f63a79ff700 1 -- 172.21.15.84:0/512215167 wait complete. 2023-07-06T21:41:51.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.605+0000 7f63a79ff700 1 Processor -- start 2023-07-06T21:41:51.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.605+0000 7f63a79ff700 1 -- start start 2023-07-06T21:41:51.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a79ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a00b1a80 0x7f63a00c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a79ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b81c0 0x7f63a01476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a79ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a01492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a79ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01f2160 con 0x7f63a00cfd10 2023-07-06T21:41:51.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a79ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00c6c60 con 0x7f63a00b1a80 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a79ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0148230 con 0x7f63a00b81c0 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a5f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a01492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a5f9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a01492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49408/0 (socket says 172.21.15.84:49408) 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a5f9c700 1 -- 172.21.15.84:0/2673798991 learned_addr learned my addr 172.21.15.84:0/2673798991 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a579b700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a00b1a80 0x7f63a00c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.606+0000 7f63a4f9a700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b81c0 0x7f63a01476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a5f9c700 1 -- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b81c0 msgr2=0x7f63a01476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a5f9c700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b81c0 0x7f63a01476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a5f9c700 1 -- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a00b1a80 msgr2=0x7f63a00c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a5f9c700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a00b1a80 0x7f63a00c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a5f9c700 1 -- 172.21.15.84:0/2673798991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009550 con 0x7f63a00cfd10 2023-07-06T21:41:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a5f9c700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a01492d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f639800b9e0 tx=0x7f639800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:51.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800c9b0 con 0x7f63a00cfd10 2023-07-06T21:41:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639800cb30 con 0x7f63a00cfd10 2023-07-06T21:41:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a0148510 con 0x7f63a00cfd10 2023-07-06T21:41:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398014440 con 0x7f63a00cfd10 2023-07-06T21:41:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.607+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a0148ad0 con 0x7f63a00cfd10 2023-07-06T21:41:51.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.608+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a004e1d0 con 0x7f63a00cfd10 2023-07-06T21:41:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.609+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f639801d3b0 con 0x7f63a00cfd10 2023-07-06T21:41:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.610+0000 7f6395ffb700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f638406acb0 0x7f638406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.610+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6398006e40 con 0x7f63a00cfd10 2023-07-06T21:41:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.611+0000 7f63a579b700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f638406acb0 0x7f638406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.611+0000 7f63a579b700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f638406acb0 0x7f638406d160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f639c004f30 tx=0x7f639c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.615+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6398054240 con 0x7f63a00cfd10 2023-07-06T21:41:51.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.813+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f63a00b4f30 con 0x7f63a00cfd10 2023-07-06T21:41:51.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.813+0000 7f6395ffb700 1 -- 172.21.15.84:0/2673798991 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6398019020 con 0x7f63a00cfd10 2023-07-06T21:41:51.816 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:41:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f638406acb0 msgr2=0x7f638406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f638406acb0 0x7f638406d160 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f639c004f30 tx=0x7f639c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 msgr2=0x7f63a01492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a01492d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f639800b9e0 tx=0x7f639800bda0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 shutdown_connections 2023-07-06T21:41:51.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f638406acb0 0x7f638406d160 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f63a00cfd10 0x7f63a01492d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a00b1a80 0x7f63a00c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 --2- 172.21.15.84:0/2673798991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63a00b81c0 0x7f63a01476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.815+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 >> 172.21.15.84:0/2673798991 conn(0x7f63a006b2b0 msgr2=0x7f63a015c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.816+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 shutdown_connections 2023-07-06T21:41:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:51.816+0000 7f63a79ff700 1 -- 172.21.15.84:0/2673798991 wait complete. 2023-07-06T21:41:51.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:41:52.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.032+0000 7f2c4f7ff700 1 Processor -- start 2023-07-06T21:41:52.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.032+0000 7f2c4f7ff700 1 -- start start 2023-07-06T21:41:52.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.033+0000 7f2c4f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.033+0000 7f2c4f7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2c480ceda0 0x7f2c480cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.033+0000 7f2c4f7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480d1830 con 0x7f2c480b1a80 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.033+0000 7f2c4f7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c480cf330 con 0x7f2c480ceda0 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.033+0000 7f2c4f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480d1270 con 0x7f2c480b81c0 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.033+0000 7f2c4cd9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2c480b1a80 0x7f2c4815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58640/0 (socket says 172.21.15.84:58640) 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4cd9a700 1 -- 172.21.15.84:0/2086219069 learned_addr learned my addr 172.21.15.84:0/2086219069 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:52.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4dd9c700 1 --2- 172.21.15.84:0/2086219069 >> v2:172.21.15.84:3300/0 conn(0x7f2c480ceda0 0x7f2c480cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4d59b700 1 --2- 172.21.15.84:0/2086219069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4dd9c700 1 -- 172.21.15.84:0/2086219069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 msgr2=0x7f2c480cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c3ffff700 1 -- 172.21.15.84:0/2086219069 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892038 0 0) 0x7f2c480d1830 con 0x7f2c480b1a80 2023-07-06T21:41:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4dd9c700 1 --2- 172.21.15.84:0/2086219069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4dd9c700 1 -- 172.21.15.84:0/2086219069 >> v1:172.21.15.84:6789/0 conn(0x7f2c480b1a80 legacy=0x7f2c4815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4dd9c700 1 -- 172.21.15.84:0/2086219069 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c480d1550 con 0x7f2c480ceda0 2023-07-06T21:41:52.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c4dd9c700 1 --2- 172.21.15.84:0/2086219069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480ceda0 0x7f2c480cfb40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009d90 tx=0x7f2c4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26939790ad3cc1a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.034+0000 7f2c3ffff700 1 -- 172.21.15.84:0/2086219069 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000e070 con 0x7f2c480ceda0 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c3ffff700 1 -- 172.21.15.84:0/2086219069 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4000bb20 con 0x7f2c480ceda0 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c3ffff700 1 -- 172.21.15.84:0/2086219069 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c40012430 con 0x7f2c480ceda0 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/2086219069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480ceda0 msgr2=0x7f2c480cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/2086219069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480ceda0 0x7f2c480cfb40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f2c40009d90 tx=0x7f2c4000b330 comp rx=0 tx=0).stop 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/2086219069 shutdown_connections 2023-07-06T21:41:52.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/2086219069 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480ceda0 0x7f2c480cfb40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/2086219069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/2086219069 >> 172.21.15.84:0/2086219069 conn(0x7f2c4806b2b0 msgr2=0x7f2c4806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/2086219069 shutdown_connections 2023-07-06T21:41:52.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.035+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/2086219069 wait complete. 2023-07-06T21:41:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.037+0000 7f2c4f7ff700 1 Processor -- start 2023-07-06T21:41:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.037+0000 7f2c4f7ff700 1 -- start start 2023-07-06T21:41:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4f7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 0x7f2c481f1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4f7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4f7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c480ceda0 0x7f2c480cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4f7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481f1c80 con 0x7f2c480b1a80 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4f7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481e3e10 con 0x7f2c480ceda0 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4f7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480cb880 con 0x7f2c480b81c0 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4d59b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 0x7f2c481f1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4d59b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 0x7f2c481f1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49420/0 (socket says 172.21.15.84:49420) 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4d59b700 1 -- 172.21.15.84:0/622677086 learned_addr learned my addr 172.21.15.84:0/622677086 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4cd9a700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4dd9c700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c480ceda0 0x7f2c480cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4d59b700 1 -- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 msgr2=0x7f2c480cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4d59b700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.038+0000 7f2c4d59b700 1 -- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c480ceda0 msgr2=0x7f2c480cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c4d59b700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c480ceda0 0x7f2c480cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c4d59b700 1 -- 172.21.15.84:0/622677086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c40009a70 con 0x7f2c480b1a80 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c4cd9a700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c4d59b700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 0x7f2c481f1740 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9b0 tx=0x7f2c4400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44009710 con 0x7f2c480b1a80 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4400c850 con 0x7f2c480b1a80 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c480cbb60 con 0x7f2c480b1a80 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400cbc0 con 0x7f2c480b1a80 2023-07-06T21:41:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.039+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c480cc120 con 0x7f2c480b1a80 2023-07-06T21:41:52.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.040+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c480b4cf0 con 0x7f2c480b1a80 2023-07-06T21:41:52.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.041+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2c4400cd20 con 0x7f2c480b1a80 2023-07-06T21:41:52.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.043+0000 7f2c3dffb700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2c2c06ace0 0x7f2c2c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.043+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2c44089690 con 0x7f2c480b1a80 2023-07-06T21:41:52.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.043+0000 7f2c4cd9a700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2c2c06ace0 0x7f2c2c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.043+0000 7f2c4cd9a700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2c2c06ace0 0x7f2c2c06d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2c480d31b0 tx=0x7f2c38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.047+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2c440580d0 con 0x7f2c480b1a80 2023-07-06T21:41:52.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.247+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c4804e1d0 con 0x7f2c480b1a80 2023-07-06T21:41:52.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.247+0000 7f2c3dffb700 1 -- 172.21.15.84:0/622677086 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c44057c60 con 0x7f2c480b1a80 2023-07-06T21:41:52.250 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:52.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2c2c06ace0 msgr2=0x7f2c2c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2c2c06ace0 0x7f2c2c06d190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2c480d31b0 tx=0x7f2c38009040 comp rx=0 tx=0).stop 2023-07-06T21:41:52.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 msgr2=0x7f2c481f1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 0x7f2c481f1740 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400e9b0 tx=0x7f2c4400ed70 comp rx=0 tx=0).stop 2023-07-06T21:41:52.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 shutdown_connections 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2c2c06ace0 0x7f2c2c06d190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c480ceda0 0x7f2c480cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c480b1a80 0x7f2c481f1740 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 --2- 172.21.15.84:0/622677086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c480b81c0 0x7f2c480cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 >> 172.21.15.84:0/622677086 conn(0x7f2c4806b2b0 msgr2=0x7f2c480b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 shutdown_connections 2023-07-06T21:41:52.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.250+0000 7f2c4f7ff700 1 -- 172.21.15.84:0/622677086 wait complete. 2023-07-06T21:41:52.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:41:52.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.465+0000 7f41227ff700 1 Processor -- start 2023-07-06T21:41:52.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.466+0000 7f41227ff700 1 -- start start 2023-07-06T21:41:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f41227ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b1a80 0x7f411c0d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f41227ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c0d1830 con 0x7f411c0b81c0 2023-07-06T21:41:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f41227ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c0cf330 con 0x7f411c0cfd10 2023-07-06T21:41:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f41227ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0d1270 con 0x7f411c0b1a80 2023-07-06T21:41:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f411bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f411c0b81c0 0x7f411c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58650/0 (socket says 172.21.15.84:58650) 2023-07-06T21:41:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f411bfff700 1 -- 172.21.15.84:0/358604617 learned_addr learned my addr 172.21.15.84:0/358604617 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f4120d9c700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b1a80 0x7f411c0d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180271722 0 0) 0x7f411c0d1830 con 0x7f411c0b81c0 2023-07-06T21:41:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.467+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41080035b0 con 0x7f411c0b81c0 2023-07-06T21:41:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324555864 0 0) 0x7f411c0cf330 con 0x7f411c0cfd10 2023-07-06T21:41:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c0d1830 con 0x7f411c0cfd10 2023-07-06T21:41:52.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1976111187 0 0) 0x7f41080035b0 con 0x7f411c0b81c0 2023-07-06T21:41:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b1a80 msgr2=0x7f411c0d2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b1a80 0x7f411c0d2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 >> v1:172.21.15.107:6789/0 conn(0x7f411c0cfd10 legacy=0x7f411c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c0d1550 con 0x7f411c0b81c0 2023-07-06T21:41:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f410c001670 con 0x7f411c0b81c0 2023-07-06T21:41:52.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 >> v1:172.21.15.84:6789/0 conn(0x7f411c0b81c0 legacy=0x7f411c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4108003bc0 0x7f4108003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4108004850 0x7f4108004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108005200 0x7f410800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0d1270 con 0x7f4108004850 2023-07-06T21:41:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410800bfd0 con 0x7f4108005200 2023-07-06T21:41:52.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f410800c190 con 0x7f4108003bc0 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f4120d9c700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b1a80 0x7f411c0d2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f4113fff700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4108004850 0x7f4108004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f411bfff700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4108003bc0 0x7f4108003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.468+0000 7f4120d9c700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108005200 0x7f410800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f4113fff700 1 -- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4108003bc0 msgr2=0x7f4108003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f4113fff700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4108003bc0 0x7f4108003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f4113fff700 1 -- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108005200 msgr2=0x7f410800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f4113fff700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108005200 0x7f410800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f4113fff700 1 -- 172.21.15.84:0/358604617 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080035b0 con 0x7f4108004850 2023-07-06T21:41:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f4113fff700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4108004850 0x7f4108004cc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f411c0d0100 tx=0x7f4108005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d2e0fdb01cca7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104001430 con 0x7f4108004850 2023-07-06T21:41:52.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41040050d0 con 0x7f4108004850 2023-07-06T21:41:52.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.469+0000 7f411b7fe700 1 -- 172.21.15.84:0/358604617 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104005480 con 0x7f4108004850 2023-07-06T21:41:52.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f41227ff700 1 -- 172.21.15.84:0/358604617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4108004850 msgr2=0x7f4108004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f41227ff700 1 --2- 172.21.15.84:0/358604617 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4108004850 0x7f4108004cc0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f411c0d0100 tx=0x7f4108005880 comp rx=0 tx=0).stop 2023-07-06T21:41:52.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f4120d9c700 1 -- 172.21.15.84:0/358604617 reap_dead start 2023-07-06T21:41:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f41227ff700 1 -- 172.21.15.84:0/358604617 shutdown_connections 2023-07-06T21:41:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f41227ff700 1 -- 172.21.15.84:0/358604617 >> 172.21.15.84:0/358604617 conn(0x7f411c06b2b0 msgr2=0x7f411c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f41227ff700 1 -- 172.21.15.84:0/358604617 shutdown_connections 2023-07-06T21:41:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.470+0000 7f41227ff700 1 -- 172.21.15.84:0/358604617 wait complete. 2023-07-06T21:41:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.472+0000 7f41227ff700 1 Processor -- start 2023-07-06T21:41:52.481 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.472+0000 7f41227ff700 1 -- start start 2023-07-06T21:41:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f41227ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108004850 0x7f411c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f41227ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 0x7f411c155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.482 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f41227ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b6a70 0x7f411c0b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f41227ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0ccc80 con 0x7f411c0cc810 2023-07-06T21:41:52.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f41227ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0b7420 con 0x7f4108004850 2023-07-06T21:41:52.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f41227ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1580e0 con 0x7f411c0b6a70 2023-07-06T21:41:52.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 0x7f411c155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 0x7f411c155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49440/0 (socket says 172.21.15.84:49440) 2023-07-06T21:41:52.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 -- 172.21.15.84:0/3559885460 learned_addr learned my addr 172.21.15.84:0/3559885460 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:52.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4120d9c700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b6a70 0x7f411c0b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.484 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f411bfff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108004850 0x7f411c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 -- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b6a70 msgr2=0x7f411c0b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b6a70 0x7f411c0b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 -- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108004850 msgr2=0x7f411c155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108004850 0x7f411c155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.473+0000 7f4113fff700 1 -- 172.21.15.84:0/3559885460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c158980 con 0x7f411c0cc810 2023-07-06T21:41:52.485 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.474+0000 7f4113fff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 0x7f411c155550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f410400c3c0 tx=0x7f41040012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.474+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104006c70 con 0x7f411c0cc810 2023-07-06T21:41:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.474+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104006dd0 con 0x7f411c0cc810 2023-07-06T21:41:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.474+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c155a90 con 0x7f411c0cc810 2023-07-06T21:41:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.474+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104007260 con 0x7f411c0cc810 2023-07-06T21:41:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.474+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c158330 con 0x7f411c0cc810 2023-07-06T21:41:52.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.475+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c158b10 con 0x7f411c0cc810 2023-07-06T21:41:52.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.476+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f41040073c0 con 0x7f411c0cc810 2023-07-06T21:41:52.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.477+0000 7f41197fa700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40fc06ace0 0x7f40fc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.477+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f410408a4c0 con 0x7f411c0cc810 2023-07-06T21:41:52.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.477+0000 7f411bfff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40fc06ace0 0x7f40fc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.488 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.478+0000 7f411bfff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40fc06ace0 0x7f40fc06d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f411c0c6470 tx=0x7f410c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.489 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.481+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f41040554f0 con 0x7f411c0cc810 2023-07-06T21:41:52.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.683+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f411c0ce150 con 0x7f411c0cc810 2023-07-06T21:41:52.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.683+0000 7f41197fa700 1 -- 172.21.15.84:0/3559885460 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4104058b10 con 0x7f411c0cc810 2023-07-06T21:41:52.686 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:41:52.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40fc06ace0 msgr2=0x7f40fc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40fc06ace0 0x7f40fc06d190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f411c0c6470 tx=0x7f410c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:52.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 msgr2=0x7f411c155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 0x7f411c155550 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f410400c3c0 tx=0x7f41040012d0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 shutdown_connections 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f40fc06ace0 0x7f40fc06d190 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f411c0b6a70 0x7f411c0b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4108004850 0x7f411c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 --2- 172.21.15.84:0/3559885460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f411c0cc810 0x7f411c155550 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 >> 172.21.15.84:0/3559885460 conn(0x7f411c06b2b0 msgr2=0x7f411c0d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 shutdown_connections 2023-07-06T21:41:52.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.686+0000 7f41227ff700 1 -- 172.21.15.84:0/3559885460 wait complete. 2023-07-06T21:41:52.705 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get data_new pg_num 2023-07-06T21:41:52.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.898+0000 7f9f7e1ff700 1 Processor -- start 2023-07-06T21:41:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.898+0000 7f9f7e1ff700 1 -- start start 2023-07-06T21:41:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.899+0000 7f9f7e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b81c0 0x7f9f780cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.899+0000 7f9f7e1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f780ceda0 0x7f9f780cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.899+0000 7f9f7e1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f780d1830 con 0x7f9f780b1a80 2023-07-06T21:41:52.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.899+0000 7f9f7e1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f780cf330 con 0x7f9f780ceda0 2023-07-06T21:41:52.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.899+0000 7f9f7e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f780d1270 con 0x7f9f780b81c0 2023-07-06T21:41:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f780ceda0 0x7f9f780cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9f780ceda0 0x7f9f780cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41884/0 (socket says 172.21.15.84:41884) 2023-07-06T21:41:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 -- 172.21.15.84:0/3067530396 learned_addr learned my addr 172.21.15.84:0/3067530396 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f777fe700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b81c0 0x7f9f780cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 -- 172.21.15.84:0/3067530396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b81c0 msgr2=0x7f9f780cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:52.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b81c0 0x7f9f780cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 -- 172.21.15.84:0/3067530396 >> v1:172.21.15.107:6789/0 conn(0x7f9f780b1a80 legacy=0x7f9f7815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 -- 172.21.15.84:0/3067530396 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f780d1550 con 0x7f9f780ceda0 2023-07-06T21:41:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f777fe700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b81c0 0x7f9f780cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:52.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f77fff700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780ceda0 0x7f9f780cfb40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009e50 tx=0x7f9f7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572cb87a15552148 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.900+0000 7f9f767fc700 1 -- 172.21.15.84:0/3067530396 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000e070 con 0x7f9f780ceda0 2023-07-06T21:41:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f767fc700 1 -- 172.21.15.84:0/3067530396 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7000b9c0 con 0x7f9f780ceda0 2023-07-06T21:41:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f767fc700 1 -- 172.21.15.84:0/3067530396 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000bd70 con 0x7f9f780ceda0 2023-07-06T21:41:52.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/3067530396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780ceda0 msgr2=0x7f9f780cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780ceda0 0x7f9f780cfb40 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009e50 tx=0x7f9f7000b330 comp rx=0 tx=0).stop 2023-07-06T21:41:52.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/3067530396 shutdown_connections 2023-07-06T21:41:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780ceda0 0x7f9f780cfb40 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/3067530396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b81c0 0x7f9f780cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/3067530396 >> 172.21.15.84:0/3067530396 conn(0x7f9f7806b2b0 msgr2=0x7f9f7806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/3067530396 shutdown_connections 2023-07-06T21:41:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.901+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/3067530396 wait complete. 2023-07-06T21:41:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.903+0000 7f9f7e1ff700 1 Processor -- start 2023-07-06T21:41:52.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.903+0000 7f9f7e1ff700 1 -- start start 2023-07-06T21:41:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f7e1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b1a80 0x7f9f78155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f7e1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 0x7f9f780b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f7e1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f780ceda0 0x7f9f780b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f7e1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78157270 con 0x7f9f780b81c0 2023-07-06T21:41:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f7e1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f78159ae0 con 0x7f9f780ceda0 2023-07-06T21:41:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f7e1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f781591f0 con 0x7f9f780b1a80 2023-07-06T21:41:52.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 0x7f9f780b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f76ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 0x7f9f780b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41890/0 (socket says 172.21.15.84:41890) 2023-07-06T21:41:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f76ffd700 1 -- 172.21.15.84:0/1982987718 learned_addr learned my addr 172.21.15.84:0/1982987718 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f777fe700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b1a80 0x7f9f78155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.904+0000 7f9f77fff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f780ceda0 0x7f9f780b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f76ffd700 1 -- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b1a80 msgr2=0x7f9f78155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f76ffd700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b1a80 0x7f9f78155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f76ffd700 1 -- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f780ceda0 msgr2=0x7f9f780b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f76ffd700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f780ceda0 0x7f9f780b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f76ffd700 1 -- 172.21.15.84:0/1982987718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70009b30 con 0x7f9f780b81c0 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f76ffd700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 0x7f9f780b3a40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c510 tx=0x7f9f6c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0052d0 con 0x7f9f780b81c0 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c005430 con 0x7f9f780b81c0 2023-07-06T21:41:52.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f78156f90 con 0x7f9f780b81c0 2023-07-06T21:41:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c0038c0 con 0x7f9f780b81c0 2023-07-06T21:41:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.905+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f78159770 con 0x7f9f780b81c0 2023-07-06T21:41:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.906+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f7804e1d0 con 0x7f9f780b81c0 2023-07-06T21:41:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.907+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9f6c003a20 con 0x7f9f780b81c0 2023-07-06T21:41:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.908+0000 7f9f5ffff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9f6406acb0 0x7f9f6406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:52.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.908+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9f6c0889f0 con 0x7f9f780b81c0 2023-07-06T21:41:52.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.908+0000 7f9f777fe700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9f6406acb0 0x7f9f6406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:52.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.909+0000 7f9f777fe700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9f6406acb0 0x7f9f6406d160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f68006f10 tx=0x7f9f68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:52.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:52.912+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9f6c053870 con 0x7f9f780b81c0 2023-07-06T21:41:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.111+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"} v 0) v1 -- 0x7f9f780b4f30 con 0x7f9f780b81c0 2023-07-06T21:41:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.112+0000 7f9f5ffff700 1 -- 172.21.15.84:0/1982987718 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "data_new", "var": "pg_num"}]=0 v40) v1 ==== 97+0+11 (secure 0 0 0) 0x7f9f6c056e90 con 0x7f9f780b81c0 2023-07-06T21:41:53.114 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:41:53.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9f6406acb0 msgr2=0x7f9f6406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9f6406acb0 0x7f9f6406d160 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f68006f10 tx=0x7f9f68008040 comp rx=0 tx=0).stop 2023-07-06T21:41:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 msgr2=0x7f9f780b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 0x7f9f780b3a40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c510 tx=0x7f9f6c00c820 comp rx=0 tx=0).stop 2023-07-06T21:41:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 shutdown_connections 2023-07-06T21:41:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9f6406acb0 0x7f9f6406d160 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f780ceda0 0x7f9f780b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f780b1a80 0x7f9f78155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 --2- 172.21.15.84:0/1982987718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f780b81c0 0x7f9f780b3a40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 >> 172.21.15.84:0/1982987718 conn(0x7f9f7806b2b0 msgr2=0x7f9f780b75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:53.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 shutdown_connections 2023-07-06T21:41:53.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.114+0000 7f9f7e1ff700 1 -- 172.21.15.84:0/1982987718 wait complete. 2023-07-06T21:41:53.135 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:41:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.325+0000 7f57985ff700 1 Processor -- start 2023-07-06T21:41:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.326+0000 7f57985ff700 1 -- start start 2023-07-06T21:41:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f57985ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f57900b83c0 0x7f57900cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f57985ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900d0a80 con 0x7f57900cfc60 2023-07-06T21:41:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f57985ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57900b8790 con 0x7f57900d3710 2023-07-06T21:41:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f57985ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900b4e70 con 0x7f57900b83c0 2023-07-06T21:41:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f5796b9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f57900cfc60 0x7f57900d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57722/0 (socket says 172.21.15.84:57722) 2023-07-06T21:41:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f5796b9c700 1 -- 172.21.15.84:0/4279207824 learned_addr learned my addr 172.21.15.84:0/4279207824 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.327+0000 7f579639b700 1 --2- 172.21.15.84:0/4279207824 >> v2:172.21.15.107:3300/0 conn(0x7f57900b83c0 0x7f57900cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316005959 0 0) 0x7f57900d0a80 con 0x7f57900cfc60 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f577c0035b0 con 0x7f57900cfc60 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615500629 0 0) 0x7f57900b8790 con 0x7f57900d3710 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57900d0a80 con 0x7f57900d3710 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3306669010 0 0) 0x7f577c0035b0 con 0x7f57900cfc60 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 >> v2:172.21.15.107:3300/0 conn(0x7f57900b83c0 msgr2=0x7f57900cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 --2- 172.21.15.84:0/4279207824 >> v2:172.21.15.107:3300/0 conn(0x7f57900b83c0 0x7f57900cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 >> v1:172.21.15.107:6789/0 conn(0x7f57900d3710 legacy=0x7f57900cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57900b5150 con 0x7f57900cfc60 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f57880019f0 con 0x7f57900cfc60 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 >> v1:172.21.15.84:6789/0 conn(0x7f57900cfc60 legacy=0x7f57900d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c003be0 0x7f577c003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f577c004850 0x7f577c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577c005250 0x7f577c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57900b4e70 con 0x7f577c005250 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c00c160 con 0x7f577c004850 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c00c320 con 0x7f577c003be0 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.328+0000 7f5796b9c700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577c005250 0x7f577c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f579639b700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c003be0 0x7f577c003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5795b9a700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f577c004850 0x7f577c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 -- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c003be0 msgr2=0x7f577c003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c003be0 0x7f577c003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 -- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f577c004850 msgr2=0x7f577c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f577c004850 0x7f577c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 -- 172.21.15.84:0/4279207824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c0035b0 con 0x7f577c005250 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 -- 172.21.15.84:0/4279207824 reap_dead start 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5796b9c700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577c005250 0x7f577c00ba10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f579008be10 tx=0x7f577c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583c67f96aac1bba server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880072d0 con 0x7f577c005250 2023-07-06T21:41:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788007430 con 0x7f577c005250 2023-07-06T21:41:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.329+0000 7f5795399700 1 -- 172.21.15.84:0/4279207824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57880077e0 con 0x7f577c005250 2023-07-06T21:41:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 -- 172.21.15.84:0/4279207824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577c005250 msgr2=0x7f577c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577c005250 0x7f577c00ba10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f579008be10 tx=0x7f577c005880 comp rx=0 tx=0).stop 2023-07-06T21:41:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 -- 172.21.15.84:0/4279207824 shutdown_connections 2023-07-06T21:41:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 --2- 172.21.15.84:0/4279207824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f577c005250 0x7f577c00ba10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 -- 172.21.15.84:0/4279207824 >> 172.21.15.84:0/4279207824 conn(0x7f579006b350 msgr2=0x7f579006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 -- 172.21.15.84:0/4279207824 shutdown_connections 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.330+0000 7f57985ff700 1 -- 172.21.15.84:0/4279207824 wait complete. 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.332+0000 7f57985ff700 1 Processor -- start 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.332+0000 7f57985ff700 1 -- start start 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f57985ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c005250 0x7f579015f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f57985ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 0x7f5790158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f57985ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57900caa00 0x7f57900cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f57985ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790155b70 con 0x7f5790157fb0 2023-07-06T21:41:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f57985ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790158960 con 0x7f57900caa00 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f57985ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57901556e0 con 0x7f577c005250 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 0x7f5790158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 0x7f5790158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41908/0 (socket says 172.21.15.84:41908) 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 -- 172.21.15.84:0/2848016606 learned_addr learned my addr 172.21.15.84:0/2848016606 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f579639b700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c005250 0x7f579015f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5796b9c700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57900caa00 0x7f57900cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 -- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c005250 msgr2=0x7f579015f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c005250 0x7f579015f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 -- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57900caa00 msgr2=0x7f57900cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57900caa00 0x7f57900cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5795b9a700 1 -- 172.21.15.84:0/2848016606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579015f850 con 0x7f5790157fb0 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.333+0000 7f5796b9c700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57900caa00 0x7f57900cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.334+0000 7f5795b9a700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 0x7f5790158420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f578000f770 tx=0x7f578000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.334+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780005320 con 0x7f5790157fb0 2023-07-06T21:41:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.334+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57800054a0 con 0x7f5790157fb0 2023-07-06T21:41:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.334+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780006aa0 con 0x7f5790157fb0 2023-07-06T21:41:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.334+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5790154ad0 con 0x7f5790157fb0 2023-07-06T21:41:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.334+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5790155090 con 0x7f5790157fb0 2023-07-06T21:41:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.335+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5790155820 con 0x7f5790157fb0 2023-07-06T21:41:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.336+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5780006c00 con 0x7f5790157fb0 2023-07-06T21:41:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.337+0000 7f5786ffd700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f577406ace0 0x7f577406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.337+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5780088b70 con 0x7f5790157fb0 2023-07-06T21:41:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.337+0000 7f579639b700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f577406ace0 0x7f577406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.338+0000 7f579639b700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f577406ace0 0x7f577406d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f578c009be0 tx=0x7f578c009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.341+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57800539f0 con 0x7f5790157fb0 2023-07-06T21:41:53.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.581+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57900d1000 con 0x7f5790157fb0 2023-07-06T21:41:53.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.581+0000 7f5786ffd700 1 -- 172.21.15.84:0/2848016606 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f5780057010 con 0x7f5790157fb0 2023-07-06T21:41:53.584 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:53.584 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:41:04.476261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:41:53.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f577406ace0 msgr2=0x7f577406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f577406ace0 0x7f577406d190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f578c009be0 tx=0x7f578c009210 comp rx=0 tx=0).stop 2023-07-06T21:41:53.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 msgr2=0x7f5790158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 0x7f5790158420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f578000f770 tx=0x7f578000fb30 comp rx=0 tx=0).stop 2023-07-06T21:41:53.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 shutdown_connections 2023-07-06T21:41:53.586 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f577406ace0 0x7f577406d190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57900caa00 0x7f57900cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f577c005250 0x7f579015f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 --2- 172.21.15.84:0/2848016606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5790157fb0 0x7f5790158420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 >> 172.21.15.84:0/2848016606 conn(0x7f579006b350 msgr2=0x7f57900d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:53.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 shutdown_connections 2023-07-06T21:41:53.587 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.584+0000 7f57985ff700 1 -- 172.21.15.84:0/2848016606 wait complete. 2023-07-06T21:41:53.588 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-07-06T21:41:53.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:53.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.794+0000 7f7545fff700 1 Processor -- start 2023-07-06T21:41:53.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.794+0000 7f7545fff700 1 -- start start 2023-07-06T21:41:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f7545fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7540159130 0x7f75400c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f7545fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75400c5000 0x7f7540070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f7545fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540070770 0x7f75400cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f7545fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f754015d930 con 0x7f75400c5000 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f7545fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7540070bb0 con 0x7f7540159130 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f7545fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754015cd20 con 0x7f7540070770 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f753effd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75400c5000 0x7f7540070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f753effd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f75400c5000 0x7f7540070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41920/0 (socket says 172.21.15.84:41920) 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f753effd700 1 -- 172.21.15.84:0/1731455797 learned_addr learned my addr 172.21.15.84:0/1731455797 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f753ffff700 1 --2- 172.21.15.84:0/1731455797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540070770 0x7f75400cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.796+0000 7f753f7fe700 1 --2- 172.21.15.84:0/1731455797 >> v2:172.21.15.107:3300/0 conn(0x7f7540159130 0x7f75400c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753effd700 1 -- 172.21.15.84:0/1731455797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540070770 msgr2=0x7f75400cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753effd700 1 --2- 172.21.15.84:0/1731455797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540070770 0x7f75400cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753effd700 1 -- 172.21.15.84:0/1731455797 >> v2:172.21.15.107:3300/0 conn(0x7f7540159130 msgr2=0x7f75400c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753effd700 1 --2- 172.21.15.84:0/1731455797 >> v2:172.21.15.107:3300/0 conn(0x7f7540159130 0x7f75400c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753effd700 1 -- 172.21.15.84:0/1731455797 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f754015d000 con 0x7f75400c5000 2023-07-06T21:41:53.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753effd700 1 --2- 172.21.15.84:0/1731455797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f7540070230 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7540067040 tx=0x7f7534009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7723c131bc35fc8 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753e7fc700 1 -- 172.21.15.84:0/1731455797 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534011830 con 0x7f75400c5000 2023-07-06T21:41:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753e7fc700 1 -- 172.21.15.84:0/1731455797 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534011990 con 0x7f75400c5000 2023-07-06T21:41:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.797+0000 7f753e7fc700 1 -- 172.21.15.84:0/1731455797 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534011d20 con 0x7f75400c5000 2023-07-06T21:41:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 -- 172.21.15.84:0/1731455797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 msgr2=0x7f7540070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 --2- 172.21.15.84:0/1731455797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f7540070230 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7540067040 tx=0x7f7534009bd0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 -- 172.21.15.84:0/1731455797 shutdown_connections 2023-07-06T21:41:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 --2- 172.21.15.84:0/1731455797 >> v2:172.21.15.107:3300/0 conn(0x7f7540159130 0x7f75400c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 --2- 172.21.15.84:0/1731455797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540070770 0x7f75400cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 --2- 172.21.15.84:0/1731455797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f7540070230 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 -- 172.21.15.84:0/1731455797 >> 172.21.15.84:0/1731455797 conn(0x7f75400cf3d0 msgr2=0x7f75400cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 -- 172.21.15.84:0/1731455797 shutdown_connections 2023-07-06T21:41:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.798+0000 7f7545fff700 1 -- 172.21.15.84:0/1731455797 wait complete. 2023-07-06T21:41:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.799+0000 7f7545fff700 1 Processor -- start 2023-07-06T21:41:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 -- start start 2023-07-06T21:41:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7540070770 0x7f75400b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f754014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540159130 0x7f754014b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7540147a10 con 0x7f75400c5000 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75400b3480 con 0x7f7540070770 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.800+0000 7f7545fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75400c5890 con 0x7f7540159130 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f754014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f754014fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41928/0 (socket says 172.21.15.84:41928) 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 -- 172.21.15.84:0/1819847239 learned_addr learned my addr 172.21.15.84:0/1819847239 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753f7fe700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7540070770 0x7f75400b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.806 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753ffff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540159130 0x7f754014b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 -- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540159130 msgr2=0x7f754014b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540159130 0x7f754014b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 -- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7540070770 msgr2=0x7f75400b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7540070770 0x7f75400b2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 -- 172.21.15.84:0/1819847239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534009550 con 0x7f75400c5000 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753f7fe700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7540070770 0x7f75400b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:53.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f753effd700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f754014fd30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7534014040 tx=0x7f7534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:53.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.801+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534004510 con 0x7f75400c5000 2023-07-06T21:41:53.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.802+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753401e070 con 0x7f75400c5000 2023-07-06T21:41:53.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.802+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7540151320 con 0x7f75400c5000 2023-07-06T21:41:53.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.802+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75340038e0 con 0x7f75400c5000 2023-07-06T21:41:53.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.802+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75401509f0 con 0x7f75400c5000 2023-07-06T21:41:53.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.803+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75400b46c0 con 0x7f75400c5000 2023-07-06T21:41:53.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.804+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7534003a40 con 0x7f75400c5000 2023-07-06T21:41:53.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.804+0000 7f7527fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f752c06ace0 0x7f752c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:53.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.805+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f753408a4b0 con 0x7f75400c5000 2023-07-06T21:41:53.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.805+0000 7f753f7fe700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f752c06ace0 0x7f752c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:53.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.805+0000 7f753f7fe700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f752c06ace0 0x7f752c06d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7530009bb0 tx=0x7f7530008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:53.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:53.809+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f75340553c0 con 0x7f75400c5000 2023-07-06T21:41:54.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.012+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75400ad820 con 0x7f75400c5000 2023-07-06T21:41:54.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.013+0000 7f7527fff700 1 -- 172.21.15.84:0/1819847239 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7f75340589e0 con 0x7f75400c5000 2023-07-06T21:41:54.015 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:54.016 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f752c06ace0 msgr2=0x7f752c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f752c06ace0 0x7f752c06d190 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7530009bb0 tx=0x7f7530008040 comp rx=0 tx=0).stop 2023-07-06T21:41:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 msgr2=0x7f754014fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f754014fd30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7534014040 tx=0x7f7534004750 comp rx=0 tx=0).stop 2023-07-06T21:41:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 shutdown_connections 2023-07-06T21:41:54.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f752c06ace0 0x7f752c06d190 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7540159130 0x7f754014b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7540070770 0x7f75400b2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.015+0000 7f7545fff700 1 --2- 172.21.15.84:0/1819847239 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f75400c5000 0x7f754014fd30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.016+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 >> 172.21.15.84:0/1819847239 conn(0x7f75400cf3d0 msgr2=0x7f754006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.016+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 shutdown_connections 2023-07-06T21:41:54.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.016+0000 7f7545fff700 1 -- 172.21.15.84:0/1819847239 wait complete. 2023-07-06T21:41:54.036 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T21:41:54.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:41:54.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.229+0000 7fe7c9dff700 1 Processor -- start 2023-07-06T21:41:54.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.229+0000 7fe7c9dff700 1 -- start start 2023-07-06T21:41:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c9dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe7c40d3710 0x7fe7c40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c9dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe7c40cfa90 0x7fe7c40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c9dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c40cd910 con 0x7fe7c40b83c0 2023-07-06T21:41:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c9dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40cff10 con 0x7fe7c40d3710 2023-07-06T21:41:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c9dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40d0a80 con 0x7fe7c40cfa90 2023-07-06T21:41:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c37fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe7c40b83c0 0x7fe7c40cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57732/0 (socket says 172.21.15.84:57732) 2023-07-06T21:41:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c37fe700 1 -- 172.21.15.84:0/2404417621 learned_addr learned my addr 172.21.15.84:0/2404417621 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7bbfff700 1 --2- 172.21.15.84:0/2404417621 >> v2:172.21.15.84:3300/0 conn(0x7fe7c40d3710 0x7fe7c40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c3fff700 1 --2- 172.21.15.84:0/2404417621 >> v2:172.21.15.107:3300/0 conn(0x7fe7c40cfa90 0x7fe7c40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7bbfff700 1 -- 172.21.15.84:0/2404417621 >> v2:172.21.15.107:3300/0 conn(0x7fe7c40cfa90 msgr2=0x7fe7c40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7bbfff700 1 --2- 172.21.15.84:0/2404417621 >> v2:172.21.15.107:3300/0 conn(0x7fe7c40cfa90 0x7fe7c40d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7bbfff700 1 -- 172.21.15.84:0/2404417621 >> v1:172.21.15.84:6789/0 conn(0x7fe7c40b83c0 legacy=0x7fe7c40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7bbfff700 1 -- 172.21.15.84:0/2404417621 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40b8790 con 0x7fe7c40d3710 2023-07-06T21:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.231+0000 7fe7c2ffd700 1 -- 172.21.15.84:0/2404417621 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529623734 0 0) 0x7fe7b4001110 con 0x7fe7c40b83c0 2023-07-06T21:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.232+0000 7fe7bbfff700 1 --2- 172.21.15.84:0/2404417621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40cf550 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40b1b00 tx=0x7fe7ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fbe7c94e6d834f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.232+0000 7fe7c2ffd700 1 -- 172.21.15.84:0/2404417621 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011830 con 0x7fe7c40d3710 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.232+0000 7fe7c2ffd700 1 -- 172.21.15.84:0/2404417621 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac011990 con 0x7fe7c40d3710 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.232+0000 7fe7c2ffd700 1 -- 172.21.15.84:0/2404417621 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac011d20 con 0x7fe7c40d3710 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.232+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2404417621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 msgr2=0x7fe7c40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.232+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2404417621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40cf550 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe7c40b1b00 tx=0x7fe7ac009b20 comp rx=0 tx=0).stop 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.233+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2404417621 shutdown_connections 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.233+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2404417621 >> v2:172.21.15.107:3300/0 conn(0x7fe7c40cfa90 0x7fe7c40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.233+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2404417621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40cf550 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.233+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2404417621 >> 172.21.15.84:0/2404417621 conn(0x7fe7c406b350 msgr2=0x7fe7c406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:54.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.233+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2404417621 shutdown_connections 2023-07-06T21:41:54.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.233+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2404417621 wait complete. 2023-07-06T21:41:54.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.234+0000 7fe7c9dff700 1 Processor -- start 2023-07-06T21:41:54.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.234+0000 7fe7c9dff700 1 -- start start 2023-07-06T21:41:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.235+0000 7fe7c9dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7c40b83c0 0x7fe7c4153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.235+0000 7fe7c9dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7c40cfa90 0x7fe7c40b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.235+0000 7fe7c9dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.235+0000 7fe7c9dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41f3370 con 0x7fe7c40d3710 2023-07-06T21:41:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.235+0000 7fe7c9dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c415e610 con 0x7fe7c40b83c0 2023-07-06T21:41:54.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.235+0000 7fe7c9dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c41f3090 con 0x7fe7c40cfa90 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41946/0 (socket says 172.21.15.84:41946) 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 -- 172.21.15.84:0/2019288165 learned_addr learned my addr 172.21.15.84:0/2019288165 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c37fe700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7c40b83c0 0x7fe7c4153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7bbfff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7c40cfa90 0x7fe7c40b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 -- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7c40cfa90 msgr2=0x7fe7c40b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7c40cfa90 0x7fe7c40b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 -- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7c40b83c0 msgr2=0x7fe7c4153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7c40b83c0 0x7fe7c4153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 -- 172.21.15.84:0/2019288165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0094a0 con 0x7fe7c40d3710 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7bbfff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7c40cfa90 0x7fe7c40b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c37fe700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7c40b83c0 0x7fe7c4153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c3fff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40b6f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00efc0 tx=0x7fe7bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc009ba0 con 0x7fe7c40d3710 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00f040 con 0x7fe7c40d3710 2023-07-06T21:41:54.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c40cb7d0 con 0x7fe7c40d3710 2023-07-06T21:41:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.236+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc014480 con 0x7fe7c40d3710 2023-07-06T21:41:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.237+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c40cbd90 con 0x7fe7c40d3710 2023-07-06T21:41:54.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.237+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c40cb630 con 0x7fe7c40d3710 2023-07-06T21:41:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.239+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe7bc0145e0 con 0x7fe7c40d3710 2023-07-06T21:41:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.239+0000 7fe7c0ff9700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe7a406ace0 0x7fe7a406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.239+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe7bc089ec0 con 0x7fe7c40d3710 2023-07-06T21:41:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.240+0000 7fe7c37fe700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe7a406ace0 0x7fe7a406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.240+0000 7fe7c37fe700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe7a406ace0 0x7fe7a406d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f60 tx=0x7fe7b4007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:54.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.244+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe7c40cb630 con 0x7fe7c40d3710 2023-07-06T21:41:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.484+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7c40cb630 con 0x7fe7c40d3710 2023-07-06T21:41:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.485+0000 7fe7c0ff9700 1 -- 172.21.15.84:0/2019288165 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4673 (secure 0 0 0) 0x7fe7c40cb630 con 0x7fe7c40d3710 2023-07-06T21:41:54.488 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:54.488 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:41:04.476261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:41:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe7a406ace0 msgr2=0x7fe7a406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe7a406ace0 0x7fe7a406d190 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4000f60 tx=0x7fe7b4007d30 comp rx=0 tx=0).stop 2023-07-06T21:41:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 msgr2=0x7fe7c40b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40b6f00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00efc0 tx=0x7fe7bc00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 shutdown_connections 2023-07-06T21:41:54.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe7a406ace0 0x7fe7a406d190 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe7c40d3710 0x7fe7c40b6f00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7c40b83c0 0x7fe7c4153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 --2- 172.21.15.84:0/2019288165 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7c40cfa90 0x7fe7c40b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 >> 172.21.15.84:0/2019288165 conn(0x7fe7c406b350 msgr2=0x7fe7c40d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:54.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 shutdown_connections 2023-07-06T21:41:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.487+0000 7fe7c9dff700 1 -- 172.21.15.84:0/2019288165 wait complete. 2023-07-06T21:41:54.494 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-07-06T21:41:54.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:41:54.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.700+0000 7f95f01ff700 1 Processor -- start 2023-07-06T21:41:54.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.700+0000 7f95f01ff700 1 -- start start 2023-07-06T21:41:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95f01ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95e80d3710 0x7f95e80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95f01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95f01ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e80cd910 con 0x7f95e80b83c0 2023-07-06T21:41:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95f01ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e80cff10 con 0x7f95e80d3710 2023-07-06T21:41:54.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95f01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80d0a80 con 0x7f95e80cfa90 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95edf9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f95e80b83c0 0x7f95e80cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57738/0 (socket says 172.21.15.84:57738) 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95ed79a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95e80d3710 0x7f95e80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.701+0000 7f95edf9b700 1 -- 172.21.15.84:0/2713733500 learned_addr learned my addr 172.21.15.84:0/2713733500 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ee79c700 1 --2- 172.21.15.84:0/2713733500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ed79a700 1 -- 172.21.15.84:0/2713733500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 msgr2=0x7f95e80d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ed79a700 1 --2- 172.21.15.84:0/2713733500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e80d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ed79a700 1 -- 172.21.15.84:0/2713733500 >> v1:172.21.15.84:6789/0 conn(0x7f95e80b83c0 legacy=0x7f95e80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ed79a700 1 -- 172.21.15.84:0/2713733500 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80b8790 con 0x7f95e80d3710 2023-07-06T21:41:54.706 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ed79a700 1 --2- 172.21.15.84:0/2713733500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80d3710 0x7f95e80cf550 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009840 tx=0x7f95d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bd847385900ad2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:54.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ecf99700 1 -- 172.21.15.84:0/2713733500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8011830 con 0x7f95e80d3710 2023-07-06T21:41:54.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ecf99700 1 -- 172.21.15.84:0/2713733500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d8011990 con 0x7f95e80d3710 2023-07-06T21:41:54.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.702+0000 7f95ecf99700 1 -- 172.21.15.84:0/2713733500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d8011d20 con 0x7f95e80d3710 2023-07-06T21:41:54.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 -- 172.21.15.84:0/2713733500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80d3710 msgr2=0x7f95e80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2713733500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80d3710 0x7f95e80cf550 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009840 tx=0x7f95d8009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:54.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 -- 172.21.15.84:0/2713733500 shutdown_connections 2023-07-06T21:41:54.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2713733500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e80d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2713733500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80d3710 0x7f95e80cf550 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 -- 172.21.15.84:0/2713733500 >> 172.21.15.84:0/2713733500 conn(0x7f95e806b350 msgr2=0x7f95e806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:54.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 -- 172.21.15.84:0/2713733500 shutdown_connections 2023-07-06T21:41:54.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.703+0000 7f95f01ff700 1 -- 172.21.15.84:0/2713733500 wait complete. 2023-07-06T21:41:54.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.705+0000 7f95f01ff700 1 Processor -- start 2023-07-06T21:41:54.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.705+0000 7f95f01ff700 1 -- start start 2023-07-06T21:41:54.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95f01ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 0x7f95e81502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95f01ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e8150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95f01ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95e80d3710 0x7f95e8150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95f01ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80cc650 con 0x7f95e80b83c0 2023-07-06T21:41:54.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95f01ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80cc930 con 0x7f95e80d3710 2023-07-06T21:41:54.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95f01ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80ca6c0 con 0x7f95e80cfa90 2023-07-06T21:41:54.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 0x7f95e81502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 0x7f95e81502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41956/0 (socket says 172.21.15.84:41956) 2023-07-06T21:41:54.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 -- 172.21.15.84:0/2163412213 learned_addr learned my addr 172.21.15.84:0/2163412213 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:54.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 -- 172.21.15.84:0/2163412213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 msgr2=0x7f95e8150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e8150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 -- 172.21.15.84:0/2163412213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95e80d3710 msgr2=0x7f95e8150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95e80d3710 0x7f95e8150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.706+0000 7f95edf9b700 1 -- 172.21.15.84:0/2163412213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80094a0 con 0x7f95e80b83c0 2023-07-06T21:41:54.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.707+0000 7f95edf9b700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 0x7f95e81502d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95e400c480 tx=0x7f95e400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:54.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.707+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4005d70 con 0x7f95e80b83c0 2023-07-06T21:41:54.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.707+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e40070d0 con 0x7f95e80b83c0 2023-07-06T21:41:54.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.707+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e80ca9a0 con 0x7f95e80b83c0 2023-07-06T21:41:54.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.707+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4007440 con 0x7f95e80b83c0 2023-07-06T21:41:54.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.707+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e80cc290 con 0x7f95e80b83c0 2023-07-06T21:41:54.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.708+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e80cc420 con 0x7f95e80b83c0 2023-07-06T21:41:54.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.709+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f95e40075a0 con 0x7f95e80b83c0 2023-07-06T21:41:54.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.710+0000 7f95de7fc700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95cc06ace0 0x7f95cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:54.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.710+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f95e4088970 con 0x7f95e80b83c0 2023-07-06T21:41:54.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.711+0000 7f95ed79a700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95cc06ace0 0x7f95cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:54.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.711+0000 7f95ed79a700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95cc06ace0 0x7f95cc06d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f95e80d3b20 tx=0x7f95d8010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:54.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.715+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f95e40538a0 con 0x7f95e80b83c0 2023-07-06T21:41:54.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.955+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95e80cd910 con 0x7f95e80b83c0 2023-07-06T21:41:54.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.956+0000 7f95de7fc700 1 -- 172.21.15.84:0/2163412213 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4673 (secure 0 0 0) 0x7f95e80cd910 con 0x7f95e80b83c0 2023-07-06T21:41:54.959 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:54.959 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:40:53.657594+0000","modified":"2023-07-06T21:41:04.476261+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5319},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5319":{"gid":5319,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.84:6837/687288152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":687288152},{"type":"v1","addr":"172.21.15.84:6837","nonce":687288152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6,7],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-07-06T21:41:54.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95cc06ace0 msgr2=0x7f95cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95cc06ace0 0x7f95cc06d190 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f95e80d3b20 tx=0x7f95d8010000 comp rx=0 tx=0).stop 2023-07-06T21:41:54.960 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 msgr2=0x7f95e81502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 0x7f95e81502d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f95e400c480 tx=0x7f95e400c840 comp rx=0 tx=0).stop 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 shutdown_connections 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95cc06ace0 0x7f95cc06d190 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95e80d3710 0x7f95e8150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95e80b83c0 0x7f95e81502d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 --2- 172.21.15.84:0/2163412213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95e80cfa90 0x7f95e8150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:54.961 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 >> 172.21.15.84:0/2163412213 conn(0x7f95e806b350 msgr2=0x7f95e80b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:54.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 shutdown_connections 2023-07-06T21:41:54.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:54.958+0000 7f95f01ff700 1 -- 172.21.15.84:0/2163412213 wait complete. 2023-07-06T21:41:54.962 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 16 2023-07-06T21:41:54.979 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:41:55.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.171+0000 7f2a513ff700 1 Processor -- start 2023-07-06T21:41:55.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.171+0000 7f2a513ff700 1 -- start start 2023-07-06T21:41:55.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.172+0000 7f2a513ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f2a4c0c9410 0x7f2a4c15ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.172+0000 7f2a513ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c070230 con 0x7f2a4c154910 2023-07-06T21:41:55.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.172+0000 7f2a513ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c15b2a0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.172+0000 7f2a513ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4c071390 con 0x7f2a4c0c9410 2023-07-06T21:41:55.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4affd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2a4c154910 0x7f2a4c070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57750/0 (socket says 172.21.15.84:57750) 2023-07-06T21:41:55.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4affd700 1 -- 172.21.15.84:0/4141889152 learned_addr learned my addr 172.21.15.84:0/4141889152 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:55.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4b7fe700 1 --2- 172.21.15.84:0/4141889152 >> v2:172.21.15.84:3300/0 conn(0x7f2a4c0c9410 0x7f2a4c15ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a49ffb700 1 -- 172.21.15.84:0/4141889152 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286835091 0 0) 0x7f2a4c070230 con 0x7f2a4c154910 2023-07-06T21:41:55.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a49ffb700 1 -- 172.21.15.84:0/4141889152 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a340035b0 con 0x7f2a4c154910 2023-07-06T21:41:55.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/4141889152 >> v1:172.21.15.107:6789/0 conn(0x7f2a4c15a7e0 legacy=0x7f2a4c0c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/4141889152 >> v1:172.21.15.84:6789/0 conn(0x7f2a4c154910 legacy=0x7f2a4c070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/4141889152 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c15b580 con 0x7f2a4c0c9410 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.173+0000 7f2a4b7fe700 1 --2- 172.21.15.84:0/4141889152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c15ac30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009d90 tx=0x7f2a4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ea123c9d00e20 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a49ffb700 1 -- 172.21.15.84:0/4141889152 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4400e070 con 0x7f2a4c0c9410 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a49ffb700 1 -- 172.21.15.84:0/4141889152 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4400bb20 con 0x7f2a4c0c9410 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a49ffb700 1 -- 172.21.15.84:0/4141889152 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44012430 con 0x7f2a4c0c9410 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 -- 172.21.15.84:0/4141889152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c0c9410 msgr2=0x7f2a4c15ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.178 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 --2- 172.21.15.84:0/4141889152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c15ac30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2a44009d90 tx=0x7f2a4400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:55.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 -- 172.21.15.84:0/4141889152 shutdown_connections 2023-07-06T21:41:55.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 --2- 172.21.15.84:0/4141889152 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c15ac30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 -- 172.21.15.84:0/4141889152 >> 172.21.15.84:0/4141889152 conn(0x7f2a4c0cf3d0 msgr2=0x7f2a4c0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:55.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 -- 172.21.15.84:0/4141889152 shutdown_connections 2023-07-06T21:41:55.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.174+0000 7f2a513ff700 1 -- 172.21.15.84:0/4141889152 wait complete. 2023-07-06T21:41:55.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.177+0000 7f2a513ff700 1 Processor -- start 2023-07-06T21:41:55.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.177+0000 7f2a513ff700 1 -- start start 2023-07-06T21:41:55.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a513ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c1502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a513ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a4c154910 0x7f2a4c0c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a513ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 0x7f2a4c1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a513ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b0000 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a513ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b02e0 con 0x7f2a4c0c9410 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a513ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c0b05c0 con 0x7f2a4c154910 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a4b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 0x7f2a4c1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a4b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 0x7f2a4c1526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41976/0 (socket says 172.21.15.84:41976) 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/1892823037 learned_addr learned my addr 172.21.15.84:0/1892823037 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a4affd700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c1502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.178+0000 7f2a4a7fc700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a4c154910 0x7f2a4c0c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a4c154910 msgr2=0x7f2a4c0c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4b7fe700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a4c154910 0x7f2a4c0c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a4c0c9410 msgr2=0x7f2a4c1502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4b7fe700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c1502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4b7fe700 1 -- 172.21.15.84:0/1892823037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009a70 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4a7fc700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a4c154910 0x7f2a4c0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:55.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4affd700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:55.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a4b7fe700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 0x7f2a4c1526d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06fbd0 tx=0x7f2a44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:55.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44012430 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44020070 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c0b08a0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004280 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.179+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c0b10a0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.180+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c0b0dc0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.181+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2a440043e0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.182+0000 7f2a337fe700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a3806ace0 0x7f2a3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.182+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f2a44022070 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.182+0000 7f2a4affd700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a3806ace0 0x7f2a3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.183+0000 7f2a4affd700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a3806ace0 0x7f2a3806d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0011e0 tx=0x7f2a3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:55.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.186+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2a4c0b0dc0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.382+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a4c0b0dc0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.383+0000 7f2a337fe700 1 -- 172.21.15.84:0/1892823037 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+20130 (secure 0 0 0) 0x7f2a4c0b0dc0 con 0x7f2a4c15a7e0 2023-07-06T21:41:55.386 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:41:55.386 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":40,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:41:05.478342+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:40:51.215137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-07-06T21:40:52.645406+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"data_new","create_time":"2023-07-06T21:41:03.311313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:41:55.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a3806ace0 msgr2=0x7f2a3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a3806ace0 0x7f2a3806d190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0011e0 tx=0x7f2a3c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 msgr2=0x7f2a4c1526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 0x7f2a4c1526d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c06fbd0 tx=0x7f2a44004ad0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 shutdown_connections 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2a3806ace0 0x7f2a3806d190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2a4c15a7e0 0x7f2a4c1526d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a4c0c9410 0x7f2a4c1502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 --2- 172.21.15.84:0/1892823037 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a4c154910 0x7f2a4c0c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 >> 172.21.15.84:0/1892823037 conn(0x7f2a4c0cf3d0 msgr2=0x7f2a4c153360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 shutdown_connections 2023-07-06T21:41:55.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.386+0000 7f2a513ff700 1 -- 172.21.15.84:0/1892823037 wait complete. 2023-07-06T21:41:55.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T21:41:55.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.598+0000 7fd94c9ff700 1 Processor -- start 2023-07-06T21:41:55.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.598+0000 7fd94c9ff700 1 -- start start 2023-07-06T21:41:55.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd94c9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9480cfc60 0x7fd948167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd94c9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9480cd910 con 0x7fd9480d3710 2023-07-06T21:41:55.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd94c9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9480d0a80 con 0x7fd9480b83c0 2023-07-06T21:41:55.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd94c9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9480b8790 con 0x7fd9480cfc60 2023-07-06T21:41:55.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd93ffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd9480d3710 0x7fd9480cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57766/0 (socket says 172.21.15.84:57766) 2023-07-06T21:41:55.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd93ffff700 1 -- 172.21.15.84:0/1855908329 learned_addr learned my addr 172.21.15.84:0/1855908329 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd946d9d700 1 --2- 172.21.15.84:0/1855908329 >> v2:172.21.15.107:3300/0 conn(0x7fd9480cfc60 0x7fd948167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115628278 0 0) 0x7fd930001110 con 0x7fd9480d3710 2023-07-06T21:41:55.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9340035b0 con 0x7fd9480d3710 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.600+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996218686 0 0) 0x7fd9340035b0 con 0x7fd9480d3710 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 >> v2:172.21.15.107:3300/0 conn(0x7fd9480cfc60 msgr2=0x7fd948167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 --2- 172.21.15.84:0/1855908329 >> v2:172.21.15.107:3300/0 conn(0x7fd9480cfc60 0x7fd948167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 >> v1:172.21.15.107:6789/0 conn(0x7fd9480b83c0 legacy=0x7fd9480cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480b4e70 con 0x7fd9480d3710 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fd930001800 con 0x7fd9480d3710 2023-07-06T21:41:55.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 >> v1:172.21.15.84:6789/0 conn(0x7fd9480d3710 legacy=0x7fd9480cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd934003a00 0x7fd934003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9340048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd934005150 0x7fd934014220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9480b8790 con 0x7fd9340043f0 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd934014890 con 0x7fd934003a00 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd934014a50 con 0x7fd934005150 2023-07-06T21:41:55.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd93ffff700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9340048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd946d9d700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd934005150 0x7fd934014220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd94659c700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd934003a00 0x7fd934003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd93ffff700 1 -- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd934005150 msgr2=0x7fd934014220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd93ffff700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd934005150 0x7fd934014220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd93ffff700 1 -- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd934003a00 msgr2=0x7fd934003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.601+0000 7fd93ffff700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd934003a00 0x7fd934003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.608 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd946d9d700 1 -- 172.21.15.84:0/1855908329 reap_dead start 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd93ffff700 1 -- 172.21.15.84:0/1855908329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340035b0 con 0x7fd9340043f0 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd93ffff700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9340048b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c190 tx=0x7fd934005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3e4a0ee9ec849 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000ccb0 con 0x7fd9340043f0 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93000ce10 con 0x7fd9340043f0 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd945d9b700 1 -- 172.21.15.84:0/1855908329 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd930005ba0 con 0x7fd9340043f0 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd94c9ff700 1 -- 172.21.15.84:0/1855908329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 msgr2=0x7fd9340048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9340048b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd93000c190 tx=0x7fd934005880 comp rx=0 tx=0).stop 2023-07-06T21:41:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd94c9ff700 1 -- 172.21.15.84:0/1855908329 shutdown_connections 2023-07-06T21:41:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/1855908329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9340048b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.602+0000 7fd94c9ff700 1 -- 172.21.15.84:0/1855908329 >> 172.21.15.84:0/1855908329 conn(0x7fd94806b350 msgr2=0x7fd94806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.603+0000 7fd94c9ff700 1 -- 172.21.15.84:0/1855908329 shutdown_connections 2023-07-06T21:41:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.603+0000 7fd94c9ff700 1 -- 172.21.15.84:0/1855908329 wait complete. 2023-07-06T21:41:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.604+0000 7fd94c9ff700 1 Processor -- start 2023-07-06T21:41:55.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.604+0000 7fd94c9ff700 1 -- start start 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.605+0000 7fd94c9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9480ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.605+0000 7fd94c9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd948150820 0x7fd9480cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.605+0000 7fd94c9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd948150c90 0x7fd9480b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.605+0000 7fd94c9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9480cad60 con 0x7fd9340043f0 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.605+0000 7fd94c9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9480cc090 con 0x7fd948150c90 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.605+0000 7fd94c9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9480c9d40 con 0x7fd948150820 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9480ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9480ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41990/0 (socket says 172.21.15.84:41990) 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 -- 172.21.15.84:0/289828354 learned_addr learned my addr 172.21.15.84:0/289828354 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd93ffff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd948150820 0x7fd9480cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd946d9d700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd948150c90 0x7fd9480b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 -- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd948150820 msgr2=0x7fd9480cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd948150820 0x7fd9480cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 -- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd948150c90 msgr2=0x7fd9480b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:55.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd948150c90 0x7fd9480b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 -- 172.21.15.84:0/289828354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9480ca020 con 0x7fd9340043f0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94659c700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9480ca820 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd93800f700 tx=0x7fd93800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380052e0 con 0x7fd9340043f0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd938005460 con 0x7fd9340043f0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9380038c0 con 0x7fd9340043f0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.606+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9480ca300 con 0x7fd9340043f0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.607+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd948151280 con 0x7fd9340043f0 2023-07-06T21:41:55.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.608+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9480d1ff0 con 0x7fd9340043f0 2023-07-06T21:41:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.609+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd938003a20 con 0x7fd9340043f0 2023-07-06T21:41:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.609+0000 7fd93effd700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd92806ace0 0x7fd92806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.610+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd938089270 con 0x7fd9340043f0 2023-07-06T21:41:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.610+0000 7fd93ffff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd92806ace0 0x7fd92806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:55.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.610+0000 7fd93ffff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd92806ace0 0x7fd92806d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd930001a40 tx=0x7fd930007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.614+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd938054220 con 0x7fd9340043f0 2023-07-06T21:41:55.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:55.852+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd9480d1d10 con 0x7fd9340043f0 2023-07-06T21:41:55.998 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-07-06T21:41:55.996+0000 7feff05f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T21:41:56.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.002+0000 7fd93effd700 1 -- 172.21.15.84:0/289828354 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd938057840 con 0x7fd9340043f0 2023-07-06T21:41:56.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.004+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd92806ace0 msgr2=0x7fd92806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:56.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.004+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd92806ace0 0x7fd92806d190 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd930001a40 tx=0x7fd930007000 comp rx=0 tx=0).stop 2023-07-06T21:41:56.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.004+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 msgr2=0x7fd9480ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:56.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.004+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9480ca820 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd93800f700 tx=0x7fd93800fa10 comp rx=0 tx=0).stop 2023-07-06T21:41:56.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 shutdown_connections 2023-07-06T21:41:56.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd92806ace0 0x7fd92806d190 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd948150c90 0x7fd9480b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9340043f0 0x7fd9480ca820 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 --2- 172.21.15.84:0/289828354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd948150820 0x7fd9480cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 >> 172.21.15.84:0/289828354 conn(0x7fd94806b350 msgr2=0x7fd9480b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:56.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 shutdown_connections 2023-07-06T21:41:56.010 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.005+0000 7fd94c9ff700 1 -- 172.21.15.84:0/289828354 wait complete. 2023-07-06T21:41:56.010 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T21:41:56.013 INFO:tasks.ceph.mds.a.smithi084.stderr: -4068> 2023-07-06T21:40:55.138+0000 7fbed37f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T21:41:56.027 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T21:41:56.068 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-07-06T21:41:56.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.226+0000 7f94821ff700 1 Processor -- start 2023-07-06T21:41:56.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.226+0000 7f94821ff700 1 -- start start 2023-07-06T21:41:56.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f94821ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0cfd10 0x7f947c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f94821ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f947c0b1a80 0x7f947c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f94821ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c0d1830 con 0x7f947c0b81c0 2023-07-06T21:41:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f94821ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c0cf330 con 0x7f947c0b1a80 2023-07-06T21:41:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f94821ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0d1270 con 0x7f947c0cfd10 2023-07-06T21:41:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f947bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f947c0b1a80 0x7f947c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:56.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f947b7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f947c0b81c0 0x7f947c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57788/0 (socket says 172.21.15.84:57788) 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.227+0000 7f947b7fe700 1 -- 172.21.15.84:0/2723523729 learned_addr learned my addr 172.21.15.84:0/2723523729 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947affd700 1 --2- 172.21.15.84:0/2723523729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0cfd10 0x7f947c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947bfff700 1 -- 172.21.15.84:0/2723523729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0cfd10 msgr2=0x7f947c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947bfff700 1 --2- 172.21.15.84:0/2723523729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0cfd10 0x7f947c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947bfff700 1 -- 172.21.15.84:0/2723523729 >> v1:172.21.15.84:6789/0 conn(0x7f947c0b81c0 legacy=0x7f947c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947bfff700 1 -- 172.21.15.84:0/2723523729 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0d1550 con 0x7f947c0b1a80 2023-07-06T21:41:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947a7fc700 1 -- 172.21.15.84:0/2723523729 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88872704 0 0) 0x7f947c0d1830 con 0x7f947c0b81c0 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947bfff700 1 --2- 172.21.15.84:0/2723523729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c0d28c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9474009dc0 tx=0x7f947400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1f843577c3f570 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947a7fc700 1 -- 172.21.15.84:0/2723523729 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947400e070 con 0x7f947c0b1a80 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947a7fc700 1 -- 172.21.15.84:0/2723523729 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f947400bae0 con 0x7f947c0b1a80 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.228+0000 7f947a7fc700 1 -- 172.21.15.84:0/2723523729 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f947400be90 con 0x7f947c0b1a80 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 -- 172.21.15.84:0/2723523729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 msgr2=0x7f947c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 --2- 172.21.15.84:0/2723523729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c0d28c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9474009dc0 tx=0x7f947400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:56.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 -- 172.21.15.84:0/2723523729 shutdown_connections 2023-07-06T21:41:56.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 --2- 172.21.15.84:0/2723523729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c0d28c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 --2- 172.21.15.84:0/2723523729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0cfd10 0x7f947c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 -- 172.21.15.84:0/2723523729 >> 172.21.15.84:0/2723523729 conn(0x7f947c06b2b0 msgr2=0x7f947c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:56.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 -- 172.21.15.84:0/2723523729 shutdown_connections 2023-07-06T21:41:56.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.229+0000 7f94821ff700 1 -- 172.21.15.84:0/2723523729 wait complete. 2023-07-06T21:41:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.231+0000 7f94821ff700 1 Processor -- start 2023-07-06T21:41:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.231+0000 7f94821ff700 1 -- start start 2023-07-06T21:41:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f94821ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:56.238 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f94821ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0b81c0 0x7f947c0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f94821ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f947c0cfd10 0x7f947c0d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f94821ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0cb2b0 con 0x7f947c0b1a80 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f94821ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0cb590 con 0x7f947c0cfd10 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f94821ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0cb870 con 0x7f947c0b81c0 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42026/0 (socket says 172.21.15.84:42026) 2023-07-06T21:41:56.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 -- 172.21.15.84:0/1432754294 learned_addr learned my addr 172.21.15.84:0/1432754294 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947affd700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0b81c0 0x7f947c0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947bfff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f947c0cfd10 0x7f947c0d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 -- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0b81c0 msgr2=0x7f947c0cc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0b81c0 0x7f947c0cc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 -- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f947c0cfd10 msgr2=0x7f947c0d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.232+0000 7f947b7fe700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f947c0cfd10 0x7f947c0d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:56.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f947b7fe700 1 -- 172.21.15.84:0/1432754294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474009a70 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f947b7fe700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c15daf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f946c0011e0 tx=0x7f946c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c004c80 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c004e00 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f947c0cbb50 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c007310 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.233+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c15f740 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.234+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c0cd330 con 0x7f947c0b1a80 2023-07-06T21:41:56.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.235+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f946c007470 con 0x7f947c0b1a80 2023-07-06T21:41:56.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.236+0000 7f9463fff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f946806ace0 0x7f946806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:56.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.236+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6543+0+0 (secure 0 0 0) 0x7f946c089410 con 0x7f947c0b1a80 2023-07-06T21:41:56.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.236+0000 7f947affd700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f946806ace0 0x7f946806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:56.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.237+0000 7f947affd700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f946806ace0 0x7f946806d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f947c0b2f70 tx=0x7f9470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:56.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.240+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f947c0cd330 con 0x7f947c0b1a80 2023-07-06T21:41:56.480 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:56.478+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f947c0cd330 con 0x7f947c0b1a80 2023-07-06T21:41:57.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.011+0000 7f9463fff700 1 -- 172.21.15.84:0/1432754294 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f947c0cd330 con 0x7f947c0b1a80 2023-07-06T21:41:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f946806ace0 msgr2=0x7f946806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f946806ace0 0x7f946806d190 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f947c0b2f70 tx=0x7f9470008040 comp rx=0 tx=0).stop 2023-07-06T21:41:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 msgr2=0x7f947c15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c15daf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f946c0011e0 tx=0x7f946c00cae0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 shutdown_connections 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f946806ace0 0x7f946806d190 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f947c0cfd10 0x7f947c0d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c0b1a80 0x7f947c15daf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 --2- 172.21.15.84:0/1432754294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f947c0b81c0 0x7f947c0cc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 >> 172.21.15.84:0/1432754294 conn(0x7f947c06b2b0 msgr2=0x7f947c0732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 shutdown_connections 2023-07-06T21:41:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.014+0000 7f94821ff700 1 -- 172.21.15.84:0/1432754294 wait complete. 2023-07-06T21:41:57.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T21:41:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.273+0000 7fec481ff700 1 Processor -- start 2023-07-06T21:41:57.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.273+0000 7fec481ff700 1 -- start start 2023-07-06T21:41:57.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec481ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec40160ac0 0x7fec400cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec481ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400ceef0 con 0x7fec400d0ea0 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec481ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cf450 con 0x7fec400cfe20 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec481ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec400cf120 con 0x7fec40160ac0 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec45f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fec400d0ea0 0x7fec400d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57804/0 (socket says 172.21.15.84:57804) 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec45f9b700 1 -- 172.21.15.84:0/427479447 learned_addr learned my addr 172.21.15.84:0/427479447 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.274+0000 7fec4579a700 1 --2- 172.21.15.84:0/427479447 >> v2:172.21.15.84:3300/0 conn(0x7fec40160ac0 0x7fec400cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:57.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec44f99700 1 -- 172.21.15.84:0/427479447 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299403825 0 0) 0x7fec400ceef0 con 0x7fec400d0ea0 2023-07-06T21:41:57.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec44f99700 1 -- 172.21.15.84:0/427479447 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec2c0035b0 con 0x7fec400d0ea0 2023-07-06T21:41:57.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec4579a700 1 -- 172.21.15.84:0/427479447 >> v1:172.21.15.107:6789/0 conn(0x7fec400cfe20 legacy=0x7fec4015fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec4579a700 1 -- 172.21.15.84:0/427479447 >> v1:172.21.15.84:6789/0 conn(0x7fec400d0ea0 legacy=0x7fec400d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec4579a700 1 -- 172.21.15.84:0/427479447 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400cd420 con 0x7fec40160ac0 2023-07-06T21:41:57.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec4579a700 1 --2- 172.21.15.84:0/427479447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec400cf8e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fec30009840 tx=0x7fec30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957d4bc65400dfc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:57.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec44f99700 1 -- 172.21.15.84:0/427479447 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30011830 con 0x7fec40160ac0 2023-07-06T21:41:57.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec44f99700 1 -- 172.21.15.84:0/427479447 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30011990 con 0x7fec40160ac0 2023-07-06T21:41:57.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.275+0000 7fec44f99700 1 -- 172.21.15.84:0/427479447 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30011d20 con 0x7fec40160ac0 2023-07-06T21:41:57.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 -- 172.21.15.84:0/427479447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 msgr2=0x7fec400cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 --2- 172.21.15.84:0/427479447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec400cf8e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fec30009840 tx=0x7fec30009b50 comp rx=0 tx=0).stop 2023-07-06T21:41:57.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 -- 172.21.15.84:0/427479447 shutdown_connections 2023-07-06T21:41:57.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 --2- 172.21.15.84:0/427479447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec400cf8e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 -- 172.21.15.84:0/427479447 >> 172.21.15.84:0/427479447 conn(0x7fec400715c0 msgr2=0x7fec40071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:57.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 -- 172.21.15.84:0/427479447 shutdown_connections 2023-07-06T21:41:57.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.276+0000 7fec481ff700 1 -- 172.21.15.84:0/427479447 wait complete. 2023-07-06T21:41:57.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.277+0000 7fec481ff700 1 Processor -- start 2023-07-06T21:41:57.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.278+0000 7fec481ff700 1 -- start start 2023-07-06T21:41:57.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.278+0000 7fec481ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec400cfe20 0x7fec400cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:57.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec481ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec401f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:57.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec481ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec40159350 0x7fec400b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:57.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec481ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b5d70 con 0x7fec40160ac0 2023-07-06T21:41:57.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec481ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400c92e0 con 0x7fec400cfe20 2023-07-06T21:41:57.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec481ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec401f2580 con 0x7fec40159350 2023-07-06T21:41:57.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec401f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:57.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec401f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42036/0 (socket says 172.21.15.84:42036) 2023-07-06T21:41:57.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 -- 172.21.15.84:0/1607515423 learned_addr learned my addr 172.21.15.84:0/1607515423 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:57.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec45f9b700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec400cfe20 0x7fec400cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:57.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 -- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec40159350 msgr2=0x7fec400b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4679c700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec40159350 0x7fec400b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:57.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec40159350 0x7fec400b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 -- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec400cfe20 msgr2=0x7fec400cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:57.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec400cfe20 0x7fec400cae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:57.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4579a700 1 -- 172.21.15.84:0/1607515423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300094a0 con 0x7fec40160ac0 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec45f9b700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec400cfe20 0x7fec400cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.279+0000 7fec4679c700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec40159350 0x7fec400b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.280+0000 7fec4579a700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec401f2040 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fec30009b90 tx=0x7fec30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.280+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300038a0 con 0x7fec40160ac0 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.280+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3001e070 con 0x7fec40160ac0 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.280+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec40151e50 con 0x7fec40160ac0 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.280+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30003c40 con 0x7fec40160ac0 2023-07-06T21:41:57.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.280+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec401f2a90 con 0x7fec40160ac0 2023-07-06T21:41:57.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.281+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec400b5ad0 con 0x7fec40160ac0 2023-07-06T21:41:57.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.282+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec3002d450 con 0x7fec40160ac0 2023-07-06T21:41:57.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.283+0000 7fec367fc700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec2406ace0 0x7fec2406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:57.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.283+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6543+0+0 (secure 0 0 0) 0x7fec3008a710 con 0x7fec40160ac0 2023-07-06T21:41:57.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.283+0000 7fec45f9b700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec2406ace0 0x7fec2406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:57.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.284+0000 7fec45f9b700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec2406ace0 0x7fec2406d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0010e0 tx=0x7fec3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:57.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.287+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec400b5ad0 con 0x7fec40160ac0 2023-07-06T21:41:57.483 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:57.481+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fec400b5eb0 con 0x7fec40160ac0 2023-07-06T21:41:58.019 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.016+0000 7fec367fc700 1 -- 172.21.15.84:0/1607515423 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v42) v1 ==== 181+0+0 (secure 0 0 0) 0x7fec400b5eb0 con 0x7fec40160ac0 2023-07-06T21:41:58.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec2406ace0 msgr2=0x7fec2406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec2406ace0 0x7fec2406d190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0010e0 tx=0x7fec3c009040 comp rx=0 tx=0).stop 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 msgr2=0x7fec401f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec401f2040 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fec30009b90 tx=0x7fec30004750 comp rx=0 tx=0).stop 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 shutdown_connections 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec2406ace0 0x7fec2406d190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec40159350 0x7fec400b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec400cfe20 0x7fec400cae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 --2- 172.21.15.84:0/1607515423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec40160ac0 0x7fec401f2040 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.020+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 >> 172.21.15.84:0/1607515423 conn(0x7fec400715c0 msgr2=0x7fec4015dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:58.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.021+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 shutdown_connections 2023-07-06T21:41:58.025 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.021+0000 7fec481ff700 1 -- 172.21.15.84:0/1607515423 wait complete. 2023-07-06T21:41:58.025 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-07-06T21:41:58.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T21:41:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.240+0000 7f89ebdff700 1 Processor -- start 2023-07-06T21:41:58.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.240+0000 7f89ebdff700 1 -- start start 2023-07-06T21:41:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.241+0000 7f89ebdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f89e40d0ea0 0x7f89e40d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.241+0000 7f89ebdff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e40ceef0 con 0x7f89e4160ac0 2023-07-06T21:41:58.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.241+0000 7f89ebdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e40cf450 con 0x7f89e415fee0 2023-07-06T21:41:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.241+0000 7f89ebdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40cf120 con 0x7f89e40d0ea0 2023-07-06T21:41:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.241+0000 7f89e939a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f89e4160ac0 0x7f89e40cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57818/0 (socket says 172.21.15.84:57818) 2023-07-06T21:41:58.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.241+0000 7f89e939a700 1 -- 172.21.15.84:0/523257440 learned_addr learned my addr 172.21.15.84:0/523257440 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:41:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e9b9b700 1 --2- 172.21.15.84:0/523257440 >> v2:172.21.15.107:3300/0 conn(0x7f89e40d0ea0 0x7f89e40d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660462133 0 0) 0x7f89e40ceef0 con 0x7f89e4160ac0 2023-07-06T21:41:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89d00035b0 con 0x7f89e4160ac0 2023-07-06T21:41:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573692633 0 0) 0x7f89e40cf450 con 0x7f89e415fee0 2023-07-06T21:41:58.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e40ceef0 con 0x7f89e415fee0 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e9b9b700 1 -- 172.21.15.84:0/523257440 >> v1:172.21.15.107:6789/0 conn(0x7f89e415fee0 legacy=0x7f89e41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e9b9b700 1 -- 172.21.15.84:0/523257440 >> v1:172.21.15.84:6789/0 conn(0x7f89e4160ac0 legacy=0x7f89e40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2625934413 0 0) 0x7f89d00035b0 con 0x7f89e4160ac0 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.242+0000 7f89e9b9b700 1 -- 172.21.15.84:0/523257440 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40cd420 con 0x7f89e40d0ea0 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89e9b9b700 1 --2- 172.21.15.84:0/523257440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e40d0ea0 0x7f89e40d1280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009820 tx=0x7f89e0009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc036fce1b3ba40 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4001670 con 0x7f89e40d0ea0 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0005c30 con 0x7f89e40d0ea0 2023-07-06T21:41:58.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89e8b99700 1 -- 172.21.15.84:0/523257440 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0004030 con 0x7f89e40d0ea0 2023-07-06T21:41:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 -- 172.21.15.84:0/523257440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e40d0ea0 msgr2=0x7f89e40d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 --2- 172.21.15.84:0/523257440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e40d0ea0 0x7f89e40d1280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f89e0009820 tx=0x7f89e0009b30 comp rx=0 tx=0).stop 2023-07-06T21:41:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 -- 172.21.15.84:0/523257440 shutdown_connections 2023-07-06T21:41:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 --2- 172.21.15.84:0/523257440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e40d0ea0 0x7f89e40d1280 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 -- 172.21.15.84:0/523257440 >> 172.21.15.84:0/523257440 conn(0x7f89e40715c0 msgr2=0x7f89e4071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:58.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 -- 172.21.15.84:0/523257440 shutdown_connections 2023-07-06T21:41:58.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.243+0000 7f89ebdff700 1 -- 172.21.15.84:0/523257440 wait complete. 2023-07-06T21:41:58.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.245+0000 7f89ebdff700 1 Processor -- start 2023-07-06T21:41:58.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.245+0000 7f89ebdff700 1 -- start start 2023-07-06T21:41:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89ebdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 0x7f89e40cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89ebdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e4160ac0 0x7f89e40b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89ebdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e414d390 0x7f89e40b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89ebdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40b8170 con 0x7f89e415fee0 2023-07-06T21:41:58.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89ebdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e40b7e90 con 0x7f89e4160ac0 2023-07-06T21:41:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89ebdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e41e6290 con 0x7f89e414d390 2023-07-06T21:41:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89e9b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 0x7f89e40cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89e9b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 0x7f89e40cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42038/0 (socket says 172.21.15.84:42038) 2023-07-06T21:41:58.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.246+0000 7f89e9b9b700 1 -- 172.21.15.84:0/2165772338 learned_addr learned my addr 172.21.15.84:0/2165772338 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e939a700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e4160ac0 0x7f89e40b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89ea39c700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e414d390 0x7f89e40b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:58.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e9b9b700 1 -- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e414d390 msgr2=0x7f89e40b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e9b9b700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e414d390 0x7f89e40b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e9b9b700 1 -- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e4160ac0 msgr2=0x7f89e40b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e9b9b700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e4160ac0 0x7f89e40b6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:58.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e9b9b700 1 -- 172.21.15.84:0/2165772338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e0009430 con 0x7f89e415fee0 2023-07-06T21:41:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89e9b9b700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 0x7f89e40cc830 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f89e0014040 tx=0x7f89e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0003c30 con 0x7f89e415fee0 2023-07-06T21:41:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0003d90 con 0x7f89e415fee0 2023-07-06T21:41:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e41e6570 con 0x7f89e415fee0 2023-07-06T21:41:58.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e001d650 con 0x7f89e415fee0 2023-07-06T21:41:58.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.247+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e414cfb0 con 0x7f89e415fee0 2023-07-06T21:41:58.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.248+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e41e6a90 con 0x7f89e415fee0 2023-07-06T21:41:58.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.249+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f89e001d7b0 con 0x7f89e415fee0 2023-07-06T21:41:58.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.250+0000 7f89da7fc700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f89c806ace0 0x7f89c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:58.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.250+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6105+0+0 (secure 0 0 0) 0x7f89e008aa10 con 0x7f89e415fee0 2023-07-06T21:41:58.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.250+0000 7f89e939a700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f89c806ace0 0x7f89c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:58.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.251+0000 7f89e939a700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f89c806ace0 0x7f89c806d190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ccd70 tx=0x7f89d40014b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:58.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.255+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f89e41e6a90 con 0x7f89e415fee0 2023-07-06T21:41:58.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:58.459+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f89e404e1d0 con 0x7f89e415fee0 2023-07-06T21:41:59.028 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.025+0000 7f89da7fc700 1 -- 172.21.15.84:0/2165772338 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v43) v1 ==== 169+0+0 (secure 0 0 0) 0x7f89e0055b20 con 0x7f89e415fee0 2023-07-06T21:41:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.030+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f89c806ace0 msgr2=0x7f89c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.030+0000 7f89ebdff700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f89c806ace0 0x7f89c806d190 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ccd70 tx=0x7f89d40014b0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.030+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 msgr2=0x7f89e40cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.030+0000 7f89ebdff700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 0x7f89e40cc830 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f89e0014040 tx=0x7f89e0004750 comp rx=0 tx=0).stop 2023-07-06T21:41:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 shutdown_connections 2023-07-06T21:41:59.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f89c806ace0 0x7f89c806d190 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89e414d390 0x7f89e40b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89e415fee0 0x7f89e40cc830 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 --2- 172.21.15.84:0/2165772338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89e4160ac0 0x7f89e40b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 >> 172.21.15.84:0/2165772338 conn(0x7f89e40715c0 msgr2=0x7f89e40d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:59.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 shutdown_connections 2023-07-06T21:41:59.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.031+0000 7f89ebdff700 1 -- 172.21.15.84:0/2165772338 wait complete. 2023-07-06T21:41:59.035 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-07-06T21:41:59.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2023-07-06T21:41:59.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.254+0000 7f92f0bff700 1 Processor -- start 2023-07-06T21:41:59.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.255+0000 7f92f0bff700 1 -- start start 2023-07-06T21:41:59.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92f0bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:59.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92f0bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92ec0cfe20 0x7f92ec15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:59.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92f0bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ec0ceef0 con 0x7f92ec0d0ea0 2023-07-06T21:41:59.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92f0bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ec0cf450 con 0x7f92ec0cfe20 2023-07-06T21:41:59.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92f0bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0cf120 con 0x7f92ec160ac0 2023-07-06T21:41:59.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92ec0cfe20 0x7f92ec15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:59.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f92ec0cfe20 0x7f92ec15fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42050/0 (socket says 172.21.15.84:42050) 2023-07-06T21:41:59.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 -- 172.21.15.84:0/2419051811 learned_addr learned my addr 172.21.15.84:0/2419051811 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:59.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92e3fff700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:59.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 -- 172.21.15.84:0/2419051811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 msgr2=0x7f92ec0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:59.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 -- 172.21.15.84:0/2419051811 >> v1:172.21.15.107:6789/0 conn(0x7f92ec0d0ea0 legacy=0x7f92ec0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:41:59.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92ead9d700 1 -- 172.21.15.84:0/2419051811 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ec0cd420 con 0x7f92ec0cfe20 2023-07-06T21:41:59.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.256+0000 7f92e3fff700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:41:59.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.257+0000 7f92ead9d700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec15fee0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009e50 tx=0x7f92e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a917d25a118996a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:59.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.257+0000 7f92e9d9b700 1 -- 172.21.15.84:0/2419051811 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e400e070 con 0x7f92ec0cfe20 2023-07-06T21:41:59.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.257+0000 7f92e9d9b700 1 -- 172.21.15.84:0/2419051811 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e4003a20 con 0x7f92ec0cfe20 2023-07-06T21:41:59.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.257+0000 7f92e9d9b700 1 -- 172.21.15.84:0/2419051811 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4003dd0 con 0x7f92ec0cfe20 2023-07-06T21:41:59.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.257+0000 7f92f0bff700 1 -- 172.21.15.84:0/2419051811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 msgr2=0x7f92ec15fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:59.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.257+0000 7f92f0bff700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec15fee0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009e50 tx=0x7f92e400b330 comp rx=0 tx=0).stop 2023-07-06T21:41:59.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.258+0000 7f92f0bff700 1 -- 172.21.15.84:0/2419051811 shutdown_connections 2023-07-06T21:41:59.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.258+0000 7f92f0bff700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec15fee0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.258+0000 7f92f0bff700 1 --2- 172.21.15.84:0/2419051811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.258+0000 7f92f0bff700 1 -- 172.21.15.84:0/2419051811 >> 172.21.15.84:0/2419051811 conn(0x7f92ec0715c0 msgr2=0x7f92ec071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:41:59.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.258+0000 7f92f0bff700 1 -- 172.21.15.84:0/2419051811 shutdown_connections 2023-07-06T21:41:59.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.258+0000 7f92f0bff700 1 -- 172.21.15.84:0/2419051811 wait complete. 2023-07-06T21:41:59.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.259+0000 7f92f0bff700 1 Processor -- start 2023-07-06T21:41:59.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.259+0000 7f92f0bff700 1 -- start start 2023-07-06T21:41:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.260+0000 7f92f0bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec14d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.260+0000 7f92f0bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec14d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.260+0000 7f92f0bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ec0b5bf0 0x7f92ec0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.260+0000 7f92f0bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0c7130 con 0x7f92ec0cfe20 2023-07-06T21:41:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.260+0000 7f92f0bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec0b65b0 con 0x7f92ec0b5bf0 2023-07-06T21:41:59.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.260+0000 7f92f0bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ec14d6b0 con 0x7f92ec160ac0 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec14d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec14d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42062/0 (socket says 172.21.15.84:42062) 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 -- 172.21.15.84:0/3319844320 learned_addr learned my addr 172.21.15.84:0/3319844320 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92e3fff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec14d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ead9d700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ec0b5bf0 0x7f92ec0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 -- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 msgr2=0x7f92ec14d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec14d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 -- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ec0b5bf0 msgr2=0x7f92ec0b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ec0b5bf0 0x7f92ec0b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 -- 172.21.15.84:0/3319844320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e4009b30 con 0x7f92ec0cfe20 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ead9d700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ec0b5bf0 0x7f92ec0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92e3fff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec14d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92ea59c700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec14d170 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c4b0 tx=0x7f92dc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:59.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.261+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011790 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.262+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0118f0 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.262+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ec0c7360 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.262+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc011ca0 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.262+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ec0c7920 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.263+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ec04e1d0 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.264+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f92dc012420 con 0x7f92ec0cfe20 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.264+0000 7f92e2ffd700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92cc06ace0 0x7f92cc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:41:59.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.265+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5691+0+0 (secure 0 0 0) 0x7f92dc088eb0 con 0x7f92ec0cfe20 2023-07-06T21:41:59.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.265+0000 7f92e3fff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92cc06ace0 0x7f92cc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:41:59.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.265+0000 7f92e3fff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92cc06ace0 0x7f92cc06d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92ec160ef0 tx=0x7f92d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:41:59.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.269+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f92dc0540c0 con 0x7f92ec0cfe20 2023-07-06T21:41:59.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:41:59.465+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92ec0d21c0 con 0x7f92ec0cfe20 2023-07-06T21:42:00.027 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.025+0000 7f92e2ffd700 1 -- 172.21.15.84:0/3319844320 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v44) v1 ==== 160+0+0 (secure 0 0 0) 0x7f92dc0576e0 con 0x7f92ec0cfe20 2023-07-06T21:42:00.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92cc06ace0 msgr2=0x7f92cc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92cc06ace0 0x7f92cc06d190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f92ec160ef0 tx=0x7f92d4008040 comp rx=0 tx=0).stop 2023-07-06T21:42:00.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 msgr2=0x7f92ec14d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec14d170 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c4b0 tx=0x7f92dc00c870 comp rx=0 tx=0).stop 2023-07-06T21:42:00.031 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 shutdown_connections 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f92cc06ace0 0x7f92cc06d190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.028+0000 7f92f0bff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92ec0b5bf0 0x7f92ec0b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.029+0000 7f92f0bff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f92ec0cfe20 0x7f92ec14d170 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.029+0000 7f92f0bff700 1 --2- 172.21.15.84:0/3319844320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92ec160ac0 0x7f92ec14d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.029+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 >> 172.21.15.84:0/3319844320 conn(0x7f92ec0715c0 msgr2=0x7f92ec0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.029+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 shutdown_connections 2023-07-06T21:42:00.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.029+0000 7f92f0bff700 1 -- 172.21.15.84:0/3319844320 wait complete. 2023-07-06T21:42:00.033 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'data_new' removed 2023-07-06T21:42:00.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T21:42:00.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.243+0000 7f71ea7ff700 1 Processor -- start 2023-07-06T21:42:00.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.243+0000 7f71ea7ff700 1 -- start start 2023-07-06T21:42:00.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.244+0000 7f71ea7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f71e4159220 0x7f71e40ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71ea7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e40cf600 con 0x7f71e40cc3d0 2023-07-06T21:42:00.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71ea7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e40ccf40 con 0x7f71e4070ab0 2023-07-06T21:42:00.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71ea7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e40c9b00 con 0x7f71e4159220 2023-07-06T21:42:00.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f71e4159220 0x7f71e40ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e37fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f71e40cc3d0 0x7f71e40703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57820/0 (socket says 172.21.15.84:57820) 2023-07-06T21:42:00.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e37fe700 1 -- 172.21.15.84:0/1475346310 learned_addr learned my addr 172.21.15.84:0/1475346310 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e3fff700 1 -- 172.21.15.84:0/1475346310 >> v1:172.21.15.107:6789/0 conn(0x7f71e4070ab0 legacy=0x7f71e406c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e3fff700 1 -- 172.21.15.84:0/1475346310 >> v1:172.21.15.84:6789/0 conn(0x7f71e40cc3d0 legacy=0x7f71e40703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e3fff700 1 -- 172.21.15.84:0/1475346310 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e40c9de0 con 0x7f71e4159220 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.245+0000 7f71e2ffd700 1 -- 172.21.15.84:0/1475346310 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481926118 0 0) 0x7f71e40cf600 con 0x7f71e40cc3d0 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71e3fff700 1 --2- 172.21.15.84:0/1475346310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4159220 0x7f71e40ca8c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009840 tx=0x7f71d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbc8c7cea654452 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71e2ffd700 1 -- 172.21.15.84:0/1475346310 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4004230 con 0x7f71e4159220 2023-07-06T21:42:00.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71e2ffd700 1 -- 172.21.15.84:0/1475346310 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4004390 con 0x7f71e4159220 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71e2ffd700 1 -- 172.21.15.84:0/1475346310 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40036a0 con 0x7f71e4159220 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1475346310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4159220 msgr2=0x7f71e40ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1475346310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4159220 0x7f71e40ca8c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009840 tx=0x7f71d4009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1475346310 shutdown_connections 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1475346310 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4159220 0x7f71e40ca8c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1475346310 >> 172.21.15.84:0/1475346310 conn(0x7f71e40cbf10 msgr2=0x7f71e40ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:00.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1475346310 shutdown_connections 2023-07-06T21:42:00.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.246+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1475346310 wait complete. 2023-07-06T21:42:00.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.248+0000 7f71ea7ff700 1 Processor -- start 2023-07-06T21:42:00.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.248+0000 7f71ea7ff700 1 -- start start 2023-07-06T21:42:00.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71ea7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 0x7f71e4154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71ea7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71e40cc3d0 0x7f71e41f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71ea7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71e4159220 0x7f71e40c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71ea7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e40c7740 con 0x7f71e4070ab0 2023-07-06T21:42:00.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71ea7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e41552d0 con 0x7f71e4159220 2023-07-06T21:42:00.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71ea7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e41f6360 con 0x7f71e40cc3d0 2023-07-06T21:42:00.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.249+0000 7f71e3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 0x7f71e4154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 0x7f71e4154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42080/0 (socket says 172.21.15.84:42080) 2023-07-06T21:42:00.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 -- 172.21.15.84:0/1347687975 learned_addr learned my addr 172.21.15.84:0/1347687975 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:00.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 -- 172.21.15.84:0/1347687975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71e40cc3d0 msgr2=0x7f71e41f5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71e40cc3d0 0x7f71e41f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 -- 172.21.15.84:0/1347687975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71e4159220 msgr2=0x7f71e40c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71e4159220 0x7f71e40c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 -- 172.21.15.84:0/1347687975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40094a0 con 0x7f71e4070ab0 2023-07-06T21:42:00.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e3fff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 0x7f71e4154d90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f71d40090b0 tx=0x7f71d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:00.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011630 con 0x7f71e4070ab0 2023-07-06T21:42:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d4011790 con 0x7f71e4070ab0 2023-07-06T21:42:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d4011b40 con 0x7f71e4070ab0 2023-07-06T21:42:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.250+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e40c6ac0 con 0x7f71e4070ab0 2023-07-06T21:42:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.251+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e40c7080 con 0x7f71e4070ab0 2023-07-06T21:42:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.252+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e404e1d0 con 0x7f71e4070ab0 2023-07-06T21:42:00.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.252+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f71d4011ca0 con 0x7f71e4070ab0 2023-07-06T21:42:00.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.253+0000 7f71e0ff9700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f71d006ace0 0x7f71d006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.254+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f71d408aa20 con 0x7f71e4070ab0 2023-07-06T21:42:00.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.254+0000 7f71e37fe700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f71d006ace0 0x7f71d006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.254+0000 7f71e37fe700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f71d006ace0 0x7f71d006d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71d80011e0 tx=0x7f71d80079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:00.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.257+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f71d4055f80 con 0x7f71e4070ab0 2023-07-06T21:42:00.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.455+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f71e40cf010 con 0x7f71e4070ab0 2023-07-06T21:42:00.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.459+0000 7f71e0ff9700 1 -- 172.21.15.84:0/1347687975 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71d405de60 con 0x7f71e4070ab0 2023-07-06T21:42:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.461+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f71d006ace0 msgr2=0x7f71d006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.461+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f71d006ace0 0x7f71d006d190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f71d80011e0 tx=0x7f71d80079b0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 msgr2=0x7f71e4154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 0x7f71e4154d90 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f71d40090b0 tx=0x7f71d40047c0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 shutdown_connections 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f71d006ace0 0x7f71d006d190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71e4159220 0x7f71e40c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f71e4070ab0 0x7f71e4154d90 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 --2- 172.21.15.84:0/1347687975 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71e40cc3d0 0x7f71e41f5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 >> 172.21.15.84:0/1347687975 conn(0x7f71e40cbf10 msgr2=0x7f71e40732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 shutdown_connections 2023-07-06T21:42:00.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.462+0000 7f71ea7ff700 1 -- 172.21.15.84:0/1347687975 wait complete. 2023-07-06T21:42:00.482 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2023-07-06T21:42:00.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.674+0000 7f8c5a9ff700 1 Processor -- start 2023-07-06T21:42:00.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.674+0000 7f8c5a9ff700 1 -- start start 2023-07-06T21:42:00.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.675+0000 7f8c5a9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c540b8530 0x7f8c540cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.675+0000 7f8c5a9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540d0990 con 0x7f8c540d3580 2023-07-06T21:42:00.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.675+0000 7f8c5a9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540cd9d0 con 0x7f8c540d1580 2023-07-06T21:42:00.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.675+0000 7f8c5a9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c540d3af0 con 0x7f8c540b8530 2023-07-06T21:42:00.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c537fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8c540d3580 0x7f8c540d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57822/0 (socket says 172.21.15.84:57822) 2023-07-06T21:42:00.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c537fe700 1 -- 172.21.15.84:0/1485966650 learned_addr learned my addr 172.21.15.84:0/1485966650 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:00.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c53fff700 1 --2- 172.21.15.84:0/1485966650 >> v2:172.21.15.107:3300/0 conn(0x7f8c540b8530 0x7f8c540cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c52ffd700 1 -- 172.21.15.84:0/1485966650 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354121346 0 0) 0x7f8c540d0990 con 0x7f8c540d3580 2023-07-06T21:42:00.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c52ffd700 1 -- 172.21.15.84:0/1485966650 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c3c0035b0 con 0x7f8c540d3580 2023-07-06T21:42:00.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c53fff700 1 -- 172.21.15.84:0/1485966650 >> v1:172.21.15.107:6789/0 conn(0x7f8c540d1580 legacy=0x7f8c5415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c52ffd700 1 -- 172.21.15.84:0/1485966650 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4227954826 0 0) 0x7f8c3c0035b0 con 0x7f8c540d3580 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c53fff700 1 -- 172.21.15.84:0/1485966650 >> v1:172.21.15.84:6789/0 conn(0x7f8c540d3580 legacy=0x7f8c540d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.676+0000 7f8c53fff700 1 -- 172.21.15.84:0/1485966650 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540cd410 con 0x7f8c540b8530 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.677+0000 7f8c53fff700 1 --2- 172.21.15.84:0/1485966650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c540cee70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009820 tx=0x7f8c44009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50eff9983253c60 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.677+0000 7f8c52ffd700 1 -- 172.21.15.84:0/1485966650 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48001670 con 0x7f8c540b8530 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.677+0000 7f8c52ffd700 1 -- 172.21.15.84:0/1485966650 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44005c30 con 0x7f8c540b8530 2023-07-06T21:42:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.677+0000 7f8c52ffd700 1 -- 172.21.15.84:0/1485966650 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c44006030 con 0x7f8c540b8530 2023-07-06T21:42:00.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.677+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1485966650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 msgr2=0x7f8c540cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.677+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1485966650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c540cee70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009820 tx=0x7f8c44009b30 comp rx=0 tx=0).stop 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.678+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1485966650 shutdown_connections 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.678+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1485966650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c540cee70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.678+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1485966650 >> 172.21.15.84:0/1485966650 conn(0x7f8c540715c0 msgr2=0x7f8c54071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.678+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1485966650 shutdown_connections 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.678+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1485966650 wait complete. 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.679+0000 7f8c5a9ff700 1 Processor -- start 2023-07-06T21:42:00.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.680+0000 7f8c5a9ff700 1 -- start start 2023-07-06T21:42:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.680+0000 7f8c5a9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c5414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.680+0000 7f8c5a9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 0x7f8c540cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.680+0000 7f8c5a9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c540d3580 0x7f8c540c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.680+0000 7f8c5a9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c7660 con 0x7f8c540d1580 2023-07-06T21:42:00.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c5a9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c7940 con 0x7f8c540b8530 2023-07-06T21:42:00.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c5a9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540c7c20 con 0x7f8c540d3580 2023-07-06T21:42:00.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 0x7f8c540cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 0x7f8c540cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42090/0 (socket says 172.21.15.84:42090) 2023-07-06T21:42:00.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 -- 172.21.15.84:0/1167965405 learned_addr learned my addr 172.21.15.84:0/1167965405 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:00.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c53fff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c5414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 -- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c540d3580 msgr2=0x7f8c540c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c58f9c700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c540d3580 0x7f8c540c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c540d3580 0x7f8c540c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 -- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 msgr2=0x7f8c5414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:00.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c5414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:00.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c537fe700 1 -- 172.21.15.84:0/1167965405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44009430 con 0x7f8c540d1580 2023-07-06T21:42:00.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.681+0000 7f8c58f9c700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c540d3580 0x7f8c540c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:00.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.682+0000 7f8c537fe700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 0x7f8c540cbe20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800c520 tx=0x7f8c4800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:00.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.682+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48004c80 con 0x7f8c540d1580 2023-07-06T21:42:00.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.682+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48004e20 con 0x7f8c540d1580 2023-07-06T21:42:00.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.682+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c540c7f00 con 0x7f8c540d1580 2023-07-06T21:42:00.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.682+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48007310 con 0x7f8c540d1580 2023-07-06T21:42:00.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.682+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c540cc360 con 0x7f8c540d1580 2023-07-06T21:42:00.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.683+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c540cc5e0 con 0x7f8c540d1580 2023-07-06T21:42:00.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.684+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8c48007470 con 0x7f8c540d1580 2023-07-06T21:42:00.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.685+0000 7f8c50ff9700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c4006ace0 0x7f8c4006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:00.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.685+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8c48088f40 con 0x7f8c540d1580 2023-07-06T21:42:00.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.685+0000 7f8c53fff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c4006ace0 0x7f8c4006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:00.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.686+0000 7f8c53fff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c4006ace0 0x7f8c4006d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8c44015040 tx=0x7f8c44007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:00.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.689+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8c480542f0 con 0x7f8c540d1580 2023-07-06T21:42:00.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:00.962+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) v1 -- 0x7f8c540c8420 con 0x7f8c540d1580 2023-07-06T21:42:01.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.040+0000 7f8c50ff9700 1 -- 172.21.15.84:0/1167965405 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v163) v1 ==== 147+0+0 (secure 0 0 0) 0x7f8c540c8420 con 0x7f8c540d1580 2023-07-06T21:42:01.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c4006ace0 msgr2=0x7f8c4006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c4006ace0 0x7f8c4006d190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8c44015040 tx=0x7f8c44007000 comp rx=0 tx=0).stop 2023-07-06T21:42:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 msgr2=0x7f8c540cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 0x7f8c540cbe20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800c520 tx=0x7f8c4800c8e0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 shutdown_connections 2023-07-06T21:42:01.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8c4006ace0 0x7f8c4006d190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c540d3580 0x7f8c540c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c540b8530 0x7f8c5414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 --2- 172.21.15.84:0/1167965405 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c540d1580 0x7f8c540cbe20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 >> 172.21.15.84:0/1167965405 conn(0x7f8c540715c0 msgr2=0x7f8c540b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:01.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 shutdown_connections 2023-07-06T21:42:01.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.043+0000 7f8c5a9ff700 1 -- 172.21.15.84:0/1167965405 wait complete. 2023-07-06T21:42:01.064 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ok 2023-07-06T21:42:01.065 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-06T21:42:01.065 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-07-06T21:42:01.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.259+0000 7f067bfff700 1 Processor -- start 2023-07-06T21:42:01.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.259+0000 7f067bfff700 1 -- start start 2023-07-06T21:42:01.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f067bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f06740d3710 0x7f06740cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:01.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f067bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740cfa90 0x7f06740d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:01.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f067bfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06740cd910 con 0x7f06740b83c0 2023-07-06T21:42:01.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f067bfff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06740cff10 con 0x7f06740d3710 2023-07-06T21:42:01.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f067bfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740d0a80 con 0x7f06740cfa90 2023-07-06T21:42:01.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f0679d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f06740b83c0 0x7f06740cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57824/0 (socket says 172.21.15.84:57824) 2023-07-06T21:42:01.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f0679d9b700 1 -- 172.21.15.84:0/1512353192 learned_addr learned my addr 172.21.15.84:0/1512353192 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:01.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.260+0000 7f067959a700 1 --2- 172.21.15.84:0/1512353192 >> v2:172.21.15.84:3300/0 conn(0x7f06740d3710 0x7f06740cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:01.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067a59c700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740cfa90 0x7f06740d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067959a700 1 -- 172.21.15.84:0/1512353192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740cfa90 msgr2=0x7f06740d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067959a700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740cfa90 0x7f06740d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067959a700 1 -- 172.21.15.84:0/1512353192 >> v1:172.21.15.84:6789/0 conn(0x7f06740b83c0 legacy=0x7f06740cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f0678d99700 1 -- 172.21.15.84:0/1512353192 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451422606 0 0) 0x7f0670001110 con 0x7f06740b83c0 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067959a700 1 -- 172.21.15.84:0/1512353192 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06740b8790 con 0x7f06740d3710 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067a59c700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740cfa90 0x7f06740d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:01.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f067959a700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06740cf550 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0664009840 tx=0x7f0664009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea0dfb28cfa5859 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f0678d99700 1 -- 172.21.15.84:0/1512353192 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664011830 con 0x7f06740d3710 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f0678d99700 1 -- 172.21.15.84:0/1512353192 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0664011990 con 0x7f06740d3710 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.261+0000 7f0678d99700 1 -- 172.21.15.84:0/1512353192 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0664011d20 con 0x7f06740d3710 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 -- 172.21.15.84:0/1512353192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 msgr2=0x7f06740cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06740cf550 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0664009840 tx=0x7f0664009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 -- 172.21.15.84:0/1512353192 shutdown_connections 2023-07-06T21:42:01.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740cfa90 0x7f06740d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 --2- 172.21.15.84:0/1512353192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06740cf550 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 -- 172.21.15.84:0/1512353192 >> 172.21.15.84:0/1512353192 conn(0x7f067406b350 msgr2=0x7f067406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:01.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 -- 172.21.15.84:0/1512353192 shutdown_connections 2023-07-06T21:42:01.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.262+0000 7f067bfff700 1 -- 172.21.15.84:0/1512353192 wait complete. 2023-07-06T21:42:01.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.264+0000 7f067bfff700 1 Processor -- start 2023-07-06T21:42:01.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.264+0000 7f067bfff700 1 -- start start 2023-07-06T21:42:01.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067bfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740b83c0 0x7f06740c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:01.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067bfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740cfa90 0x7f06740c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06741f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067bfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067415c930 con 0x7f06740d3710 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067bfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06740c6ec0 con 0x7f06740cfa90 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067bfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06740c71a0 con 0x7f06740b83c0 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06741f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06741f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42110/0 (socket says 172.21.15.84:42110) 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 -- 172.21.15.84:0/3791690693 learned_addr learned my addr 172.21.15.84:0/3791690693 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:01.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f0679d9b700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740b83c0 0x7f06740c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067959a700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740cfa90 0x7f06740c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 -- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740b83c0 msgr2=0x7f06740c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740b83c0 0x7f06740c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 -- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740cfa90 msgr2=0x7f06740c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740cfa90 0x7f06740c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067a59c700 1 -- 172.21.15.84:0/3791690693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06640094a0 con 0x7f06740d3710 2023-07-06T21:42:01.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f067959a700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740cfa90 0x7f06740c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:01.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.265+0000 7f0679d9b700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740b83c0 0x7f06740c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:01.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.266+0000 7f067a59c700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06741f3120 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f066c007e70 tx=0x7f066c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.266+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00de40 con 0x7f06740d3710 2023-07-06T21:42:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.266+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c00f040 con 0x7f06740d3710 2023-07-06T21:42:01.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.266+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06740c7480 con 0x7f06740d3710 2023-07-06T21:42:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.266+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c003c60 con 0x7f06740d3710 2023-07-06T21:42:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.266+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06740c6bb0 con 0x7f06740d3710 2023-07-06T21:42:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.267+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06740c7930 con 0x7f06740d3710 2023-07-06T21:42:01.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.268+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f066c004030 con 0x7f06740d3710 2023-07-06T21:42:01.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.269+0000 7f066a7fc700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f065806ace0 0x7f065806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:01.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.269+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f066c089270 con 0x7f06740d3710 2023-07-06T21:42:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.269+0000 7f0679d9b700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f065806ace0 0x7f065806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.270+0000 7f0679d9b700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f065806ace0 0x7f065806d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0670000f60 tx=0x7f0670007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:01.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.273+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f066c0546b0 con 0x7f06740d3710 2023-07-06T21:42:01.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:01.553+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f067406c090 con 0x7f06740d3710 2023-07-06T21:42:02.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.042+0000 7f066a7fc700 1 -- 172.21.15.84:0/3791690693 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v164) v1 ==== 137+0+0 (secure 0 0 0) 0x7f066c057cd0 con 0x7f06740d3710 2023-07-06T21:42:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f065806ace0 msgr2=0x7f065806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f065806ace0 0x7f065806d190 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0670000f60 tx=0x7f0670007d50 comp rx=0 tx=0).stop 2023-07-06T21:42:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 msgr2=0x7f06741f3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06741f3120 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f066c007e70 tx=0x7f066c00d3b0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 shutdown_connections 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f065806ace0 0x7f065806d190 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06740d3710 0x7f06741f3120 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06740b83c0 0x7f06740c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 --2- 172.21.15.84:0/3791690693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06740cfa90 0x7f06740c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 >> 172.21.15.84:0/3791690693 conn(0x7f067406b350 msgr2=0x7f06740b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:02.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 shutdown_connections 2023-07-06T21:42:02.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.045+0000 7f067bfff700 1 -- 172.21.15.84:0/3791690693 wait complete. 2023-07-06T21:42:02.066 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:02.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.258+0000 7f9e155ff700 1 Processor -- start 2023-07-06T21:42:02.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.258+0000 7f9e155ff700 1 -- start start 2023-07-06T21:42:02.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.259+0000 7f9e155ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e100b83c0 0x7f9e100cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e155ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9e100d3710 0x7f9e100cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e155ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e100d0a80 con 0x7f9e100cfa90 2023-07-06T21:42:02.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e155ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e100cff10 con 0x7f9e100b83c0 2023-07-06T21:42:02.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e155ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e100b8790 con 0x7f9e100d3710 2023-07-06T21:42:02.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e100b83c0 0x7f9e100cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9e100b83c0 0x7f9e100cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42118/0 (socket says 172.21.15.84:42118) 2023-07-06T21:42:02.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 -- 172.21.15.84:0/3987333198 learned_addr learned my addr 172.21.15.84:0/3987333198 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:02.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e07fff700 1 --2- 172.21.15.84:0/3987333198 >> v2:172.21.15.107:3300/0 conn(0x7f9e100d3710 0x7f9e100cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 -- 172.21.15.84:0/3987333198 >> v2:172.21.15.107:3300/0 conn(0x7f9e100d3710 msgr2=0x7f9e100cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:02.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 --2- 172.21.15.84:0/3987333198 >> v2:172.21.15.107:3300/0 conn(0x7f9e100d3710 0x7f9e100cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 -- 172.21.15.84:0/3987333198 >> v1:172.21.15.107:6789/0 conn(0x7f9e100cfa90 legacy=0x7f9e1015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.260+0000 7f9e0effd700 1 -- 172.21.15.84:0/3987333198 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100b4e70 con 0x7f9e100b83c0 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.261+0000 7f9e0effd700 1 --2- 172.21.15.84:0/3987333198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100b83c0 0x7f9e100cd200 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9e100b8cb0 tx=0x7f9e00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59661b5b97851d3f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.261+0000 7f9e0e7fc700 1 -- 172.21.15.84:0/3987333198 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00011830 con 0x7f9e100b83c0 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.261+0000 7f9e0e7fc700 1 -- 172.21.15.84:0/3987333198 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e00011990 con 0x7f9e100b83c0 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.261+0000 7f9e0e7fc700 1 -- 172.21.15.84:0/3987333198 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00011d20 con 0x7f9e100b83c0 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.261+0000 7f9e155ff700 1 -- 172.21.15.84:0/3987333198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100b83c0 msgr2=0x7f9e100cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.261+0000 7f9e155ff700 1 --2- 172.21.15.84:0/3987333198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100b83c0 0x7f9e100cd200 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9e100b8cb0 tx=0x7f9e00009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.262+0000 7f9e155ff700 1 -- 172.21.15.84:0/3987333198 shutdown_connections 2023-07-06T21:42:02.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.262+0000 7f9e155ff700 1 --2- 172.21.15.84:0/3987333198 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100b83c0 0x7f9e100cd200 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.262+0000 7f9e155ff700 1 --2- 172.21.15.84:0/3987333198 >> v2:172.21.15.107:3300/0 conn(0x7f9e100d3710 0x7f9e100cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.262+0000 7f9e155ff700 1 -- 172.21.15.84:0/3987333198 >> 172.21.15.84:0/3987333198 conn(0x7f9e1006b350 msgr2=0x7f9e1006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:02.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.262+0000 7f9e155ff700 1 -- 172.21.15.84:0/3987333198 shutdown_connections 2023-07-06T21:42:02.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.262+0000 7f9e155ff700 1 -- 172.21.15.84:0/3987333198 wait complete. 2023-07-06T21:42:02.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.263+0000 7f9e155ff700 1 Processor -- start 2023-07-06T21:42:02.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.263+0000 7f9e155ff700 1 -- start start 2023-07-06T21:42:02.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.264+0000 7f9e155ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e100b83c0 0x7f9e101541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.264+0000 7f9e155ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 0x7f9e100c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.264+0000 7f9e155ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e100d3710 0x7f9e100b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.264+0000 7f9e155ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100c7d50 con 0x7f9e100cfa90 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.264+0000 7f9e155ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e100c7990 con 0x7f9e100d3710 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.264+0000 7f9e155ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10154730 con 0x7f9e100b83c0 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 0x7f9e100c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 0x7f9e100c6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42124/0 (socket says 172.21.15.84:42124) 2023-07-06T21:42:02.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 -- 172.21.15.84:0/992317146 learned_addr learned my addr 172.21.15.84:0/992317146 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e0effd700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e100b83c0 0x7f9e101541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e0f7fe700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e100d3710 0x7f9e100b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 -- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e100b83c0 msgr2=0x7f9e101541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e100b83c0 0x7f9e101541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 -- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e100d3710 msgr2=0x7f9e100b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e100d3710 0x7f9e100b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 -- 172.21.15.84:0/992317146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00009550 con 0x7f9e100cfa90 2023-07-06T21:42:02.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.265+0000 7f9e07fff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 0x7f9e100c6670 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9df800e840 tx=0x7f9df800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.266+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8009590 con 0x7f9e100cfa90 2023-07-06T21:42:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.266+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8013680 con 0x7f9e100cfa90 2023-07-06T21:42:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.266+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8013a30 con 0x7f9e100cfa90 2023-07-06T21:42:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.266+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e100c70f0 con 0x7f9e100cfa90 2023-07-06T21:42:02.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.266+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e100c76b0 con 0x7f9e100cfa90 2023-07-06T21:42:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.267+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e1015b000 con 0x7f9e100cfa90 2023-07-06T21:42:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.268+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9df8013b90 con 0x7f9e100cfa90 2023-07-06T21:42:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.269+0000 7f9e077fe700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.269+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9df8089060 con 0x7f9e100cfa90 2023-07-06T21:42:02.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.269+0000 7f9e0effd700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.270+0000 7f9e0effd700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9df006ace0 0x7f9df006d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9e00000c00 tx=0x7f9e00008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:02.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.273+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9df80bb310 con 0x7f9e100cfa90 2023-07-06T21:42:02.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.469+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e100b5400 con 0x7f9e100cfa90 2023-07-06T21:42:02.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.470+0000 7f9e077fe700 1 -- 172.21.15.84:0/992317146 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14766 (secure 0 0 0) 0x7f9df8017090 con 0x7f9e100cfa90 2023-07-06T21:42:02.473 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:02.473 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":44,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:00.020744+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:6837/687288152":"2023-07-07T21:41:55.853208+0000","172.21.15.84:6835/687288152":"2023-07-07T21:41:55.853208+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:02.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9df006ace0 msgr2=0x7f9df006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9df006ace0 0x7f9df006d190 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9e00000c00 tx=0x7f9e00008030 comp rx=0 tx=0).stop 2023-07-06T21:42:02.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 msgr2=0x7f9e100c6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 0x7f9e100c6670 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9df800e840 tx=0x7f9df800eb50 comp rx=0 tx=0).stop 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 shutdown_connections 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9df006ace0 0x7f9df006d190 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e100d3710 0x7f9e100b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e100b83c0 0x7f9e101541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 --2- 172.21.15.84:0/992317146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e100cfa90 0x7f9e100c6670 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 >> 172.21.15.84:0/992317146 conn(0x7f9e1006b350 msgr2=0x7f9e1015bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:02.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 shutdown_connections 2023-07-06T21:42:02.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.473+0000 7f9e155ff700 1 -- 172.21.15.84:0/992317146 wait complete. 2023-07-06T21:42:02.494 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:02.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.687+0000 7f22ddbff700 1 Processor -- start 2023-07-06T21:42:02.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.687+0000 7f22ddbff700 1 -- start start 2023-07-06T21:42:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.688+0000 7f22ddbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80b1a80 0x7f22d80d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.688+0000 7f22ddbff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80d1830 con 0x7f22d80cfd10 2023-07-06T21:42:02.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.688+0000 7f22ddbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80cf330 con 0x7f22d80b81c0 2023-07-06T21:42:02.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.688+0000 7f22ddbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80d1270 con 0x7f22d80b1a80 2023-07-06T21:42:02.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d6ffd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f22d80cfd10 0x7f22d80d2380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:57840/0 (socket says 172.21.15.84:57840) 2023-07-06T21:42:02.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d6ffd700 1 -- 172.21.15.84:0/1114026004 learned_addr learned my addr 172.21.15.84:0/1114026004 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:02.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d7fff700 1 --2- 172.21.15.84:0/1114026004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80b1a80 0x7f22d80d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132971899 0 0) 0x7f22d80d1830 con 0x7f22d80cfd10 2023-07-06T21:42:02.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c00035b0 con 0x7f22d80cfd10 2023-07-06T21:42:02.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2642991836 0 0) 0x7f22d80cf330 con 0x7f22d80b81c0 2023-07-06T21:42:02.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d80d1830 con 0x7f22d80b81c0 2023-07-06T21:42:02.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131918937 0 0) 0x7f22c00035b0 con 0x7f22d80cfd10 2023-07-06T21:42:02.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d7fff700 1 -- 172.21.15.84:0/1114026004 >> v1:172.21.15.107:6789/0 conn(0x7f22d80b81c0 legacy=0x7f22d80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d7fff700 1 -- 172.21.15.84:0/1114026004 >> v1:172.21.15.84:6789/0 conn(0x7f22d80cfd10 legacy=0x7f22d80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.689+0000 7f22d7fff700 1 -- 172.21.15.84:0/1114026004 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80d1550 con 0x7f22d80b1a80 2023-07-06T21:42:02.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22d7fff700 1 --2- 172.21.15.84:0/1114026004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80b1a80 0x7f22d80d2a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009dc0 tx=0x7f22d000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e98d0e5ab1285ef server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:02.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000e070 con 0x7f22d80b1a80 2023-07-06T21:42:02.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d0003a20 con 0x7f22d80b1a80 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22d67fc700 1 -- 172.21.15.84:0/1114026004 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc001670 con 0x7f22d80b1a80 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22ddbff700 1 -- 172.21.15.84:0/1114026004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80b1a80 msgr2=0x7f22d80d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22ddbff700 1 --2- 172.21.15.84:0/1114026004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80b1a80 0x7f22d80d2a90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009dc0 tx=0x7f22d000b330 comp rx=0 tx=0).stop 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22ddbff700 1 -- 172.21.15.84:0/1114026004 shutdown_connections 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22ddbff700 1 --2- 172.21.15.84:0/1114026004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80b1a80 0x7f22d80d2a90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.690+0000 7f22ddbff700 1 -- 172.21.15.84:0/1114026004 >> 172.21.15.84:0/1114026004 conn(0x7f22d806b2b0 msgr2=0x7f22d806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:02.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.691+0000 7f22ddbff700 1 -- 172.21.15.84:0/1114026004 shutdown_connections 2023-07-06T21:42:02.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.691+0000 7f22ddbff700 1 -- 172.21.15.84:0/1114026004 wait complete. 2023-07-06T21:42:02.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.692+0000 7f22ddbff700 1 Processor -- start 2023-07-06T21:42:02.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.692+0000 7f22ddbff700 1 -- start start 2023-07-06T21:42:02.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22ddbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 0x7f22d81477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22ddbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d80b81c0 0x7f22d81e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22ddbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80cfd10 0x7f22d81493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22ddbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8148fb0 con 0x7f22d80b1a80 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22ddbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8147cf0 con 0x7f22d80b81c0 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22ddbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8148330 con 0x7f22d80cfd10 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.693+0000 7f22d77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 0x7f22d81477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 0x7f22d81477b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42132/0 (socket says 172.21.15.84:42132) 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 -- 172.21.15.84:0/2426682626 learned_addr learned my addr 172.21.15.84:0/2426682626 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d6ffd700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d80b81c0 0x7f22d81e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d7fff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80cfd10 0x7f22d81493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 -- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80cfd10 msgr2=0x7f22d81493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80cfd10 0x7f22d81493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 -- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d80b81c0 msgr2=0x7f22d81e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d80b81c0 0x7f22d81e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 -- 172.21.15.84:0/2426682626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009a70 con 0x7f22d80b1a80 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d7fff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80cfd10 0x7f22d81493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d6ffd700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d80b81c0 0x7f22d81e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:02.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22d77fe700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 0x7f22d81477b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c520 tx=0x7f22c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:02.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80052d0 con 0x7f22d80b1a80 2023-07-06T21:42:02.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c8005430 con 0x7f22d80b1a80 2023-07-06T21:42:02.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80038c0 con 0x7f22d80b1a80 2023-07-06T21:42:02.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.694+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d8148610 con 0x7f22d80b1a80 2023-07-06T21:42:02.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.695+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d8148bd0 con 0x7f22d80b1a80 2023-07-06T21:42:02.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.696+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d804e1d0 con 0x7f22d80b1a80 2023-07-06T21:42:02.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.697+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f22c8003a20 con 0x7f22d80b1a80 2023-07-06T21:42:02.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.697+0000 7f22bffff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f22c406ace0 0x7f22c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:02.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.697+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f22c80886c0 con 0x7f22d80b1a80 2023-07-06T21:42:02.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.698+0000 7f22d6ffd700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f22c406ace0 0x7f22c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:02.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.698+0000 7f22d6ffd700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f22c406ace0 0x7f22c406d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22d80b2a80 tx=0x7f22cc007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:02.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.702+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f22c8053b00 con 0x7f22d80b1a80 2023-07-06T21:42:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.895+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22d80b4f30 con 0x7f22d80b1a80 2023-07-06T21:42:02.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.895+0000 7f22bffff700 1 -- 172.21.15.84:0/2426682626 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22c8057120 con 0x7f22d80b1a80 2023-07-06T21:42:02.897 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:02.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f22c406ace0 msgr2=0x7f22c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f22c406ace0 0x7f22c406d190 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f22d80b2a80 tx=0x7f22cc007a80 comp rx=0 tx=0).stop 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 msgr2=0x7f22d81477b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 0x7f22d81477b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f22c800c520 tx=0x7f22c800c8e0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 shutdown_connections 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f22c406ace0 0x7f22c406d190 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22d80cfd10 0x7f22d81493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22d80b1a80 0x7f22d81477b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 --2- 172.21.15.84:0/2426682626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22d80b81c0 0x7f22d81e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 >> 172.21.15.84:0/2426682626 conn(0x7f22d806b2b0 msgr2=0x7f22d80d19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 shutdown_connections 2023-07-06T21:42:02.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:02.897+0000 7f22ddbff700 1 -- 172.21.15.84:0/2426682626 wait complete. 2023-07-06T21:42:02.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:03.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.114+0000 7f9d5afff700 1 Processor -- start 2023-07-06T21:42:03.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.115+0000 7f9d5afff700 1 -- start start 2023-07-06T21:42:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d5afff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9d540b81c0 0x7f9d540cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d5afff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d540d1830 con 0x7f9d540ceda0 2023-07-06T21:42:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d5afff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d54160d40 con 0x7f9d540b1a80 2023-07-06T21:42:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d5afff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d540cf330 con 0x7f9d540b81c0 2023-07-06T21:42:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d5959c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9d540ceda0 0x7f9d54160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45112/0 (socket says 172.21.15.84:45112) 2023-07-06T21:42:03.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d5959c700 1 -- 172.21.15.84:0/2315900636 learned_addr learned my addr 172.21.15.84:0/2315900636 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:03.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d58d9b700 1 --2- 172.21.15.84:0/2315900636 >> v2:172.21.15.107:3300/0 conn(0x7f9d540b81c0 0x7f9d540cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d537fe700 1 -- 172.21.15.84:0/2315900636 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531849368 0 0) 0x7f9d540d1830 con 0x7f9d540ceda0 2023-07-06T21:42:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.116+0000 7f9d537fe700 1 -- 172.21.15.84:0/2315900636 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d440035b0 con 0x7f9d540ceda0 2023-07-06T21:42:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d58d9b700 1 -- 172.21.15.84:0/2315900636 >> v1:172.21.15.107:6789/0 conn(0x7f9d540b1a80 legacy=0x7f9d5415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d537fe700 1 -- 172.21.15.84:0/2315900636 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 43708133 0 0) 0x7f9d440035b0 con 0x7f9d540ceda0 2023-07-06T21:42:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d58d9b700 1 -- 172.21.15.84:0/2315900636 >> v1:172.21.15.84:6789/0 conn(0x7f9d540ceda0 legacy=0x7f9d54160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d58d9b700 1 -- 172.21.15.84:0/2315900636 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d540d1270 con 0x7f9d540b81c0 2023-07-06T21:42:03.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d58d9b700 1 --2- 172.21.15.84:0/2315900636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d540cf600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009840 tx=0x7f9d48009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8be5c4570419afe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:03.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d537fe700 1 -- 172.21.15.84:0/2315900636 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0019f0 con 0x7f9d540b81c0 2023-07-06T21:42:03.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d537fe700 1 -- 172.21.15.84:0/2315900636 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d48004030 con 0x7f9d540b81c0 2023-07-06T21:42:03.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.117+0000 7f9d537fe700 1 -- 172.21.15.84:0/2315900636 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480043c0 con 0x7f9d540b81c0 2023-07-06T21:42:03.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 -- 172.21.15.84:0/2315900636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 msgr2=0x7f9d540cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 --2- 172.21.15.84:0/2315900636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d540cf600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009840 tx=0x7f9d48009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:03.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 -- 172.21.15.84:0/2315900636 shutdown_connections 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 --2- 172.21.15.84:0/2315900636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d540cf600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 -- 172.21.15.84:0/2315900636 >> 172.21.15.84:0/2315900636 conn(0x7f9d5406b2b0 msgr2=0x7f9d5406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 -- 172.21.15.84:0/2315900636 shutdown_connections 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.118+0000 7f9d5afff700 1 -- 172.21.15.84:0/2315900636 wait complete. 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.120+0000 7f9d5afff700 1 Processor -- start 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.120+0000 7f9d5afff700 1 -- start start 2023-07-06T21:42:03.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5afff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 0x7f9d541503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5afff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d5415d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d540ceda0 0x7f9d5415d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5afff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540cc650 con 0x7f9d540b1a80 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5afff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540cba40 con 0x7f9d540b81c0 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d540cbd20 con 0x7f9d540ceda0 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 0x7f9d541503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 0x7f9d541503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54014/0 (socket says 172.21.15.84:54014) 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 -- 172.21.15.84:0/4181634087 learned_addr learned my addr 172.21.15.84:0/4181634087 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5959c700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d540ceda0 0x7f9d5415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d53fff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d5415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 -- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d540ceda0 msgr2=0x7f9d5415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d540ceda0 0x7f9d5415d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 -- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 msgr2=0x7f9d5415d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d5415d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d58d9b700 1 -- 172.21.15.84:0/4181634087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480094a0 con 0x7f9d540b1a80 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d53fff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d5415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.121+0000 7f9d5959c700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d540ceda0 0x7f9d5415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.122+0000 7f9d58d9b700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 0x7f9d541503f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009b90 tx=0x7f9d48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.122+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48011830 con 0x7f9d540b1a80 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.122+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4801f070 con 0x7f9d540b1a80 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.122+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d540cc000 con 0x7f9d540b1a80 2023-07-06T21:42:03.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.122+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48011bd0 con 0x7f9d540b1a80 2023-07-06T21:42:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.122+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d540cad10 con 0x7f9d540b1a80 2023-07-06T21:42:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.123+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5404e1d0 con 0x7f9d540b1a80 2023-07-06T21:42:03.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.124+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9d48011d30 con 0x7f9d540b1a80 2023-07-06T21:42:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.125+0000 7f9d517fa700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d3806ace0 0x7f9d3806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.125+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9d48021070 con 0x7f9d540b1a80 2023-07-06T21:42:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.125+0000 7f9d53fff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d3806ace0 0x7f9d3806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.126+0000 7f9d53fff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d3806ace0 0x7f9d3806d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d540c6270 tx=0x7f9d40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:03.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.129+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9d48055f50 con 0x7f9d540b1a80 2023-07-06T21:42:03.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.324+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9d540b4f30 con 0x7f9d540b1a80 2023-07-06T21:42:03.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.325+0000 7f9d517fa700 1 -- 172.21.15.84:0/4181634087 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d4802b0a0 con 0x7f9d540b1a80 2023-07-06T21:42:03.327 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d3806ace0 msgr2=0x7f9d3806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d3806ace0 0x7f9d3806d190 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9d540c6270 tx=0x7f9d40009040 comp rx=0 tx=0).stop 2023-07-06T21:42:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 msgr2=0x7f9d541503f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 0x7f9d541503f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009b90 tx=0x7f9d48004750 comp rx=0 tx=0).stop 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 shutdown_connections 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9d3806ace0 0x7f9d3806d190 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d540ceda0 0x7f9d5415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d540b1a80 0x7f9d541503f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 --2- 172.21.15.84:0/4181634087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d540b81c0 0x7f9d5415d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 >> 172.21.15.84:0/4181634087 conn(0x7f9d5406b2b0 msgr2=0x7f9d5415c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 shutdown_connections 2023-07-06T21:42:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.327+0000 7f9d5afff700 1 -- 172.21.15.84:0/4181634087 wait complete. 2023-07-06T21:42:03.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-07-06T21:42:03.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.540+0000 7f83d07ff700 1 Processor -- start 2023-07-06T21:42:03.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.541+0000 7f83d07ff700 1 -- start start 2023-07-06T21:42:03.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83d07ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83c80b81c0 0x7f83c80cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83d07ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c80d1830 con 0x7f83c80ceda0 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83d07ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c8160d40 con 0x7f83c80b1a80 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83d07ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c80cf330 con 0x7f83c80b81c0 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83ced9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f83c80ceda0 0x7f83c8160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45120/0 (socket says 172.21.15.84:45120) 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83ced9c700 1 -- 172.21.15.84:0/4251270503 learned_addr learned my addr 172.21.15.84:0/4251270503 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.542+0000 7f83ce59b700 1 --2- 172.21.15.84:0/4251270503 >> v2:172.21.15.107:3300/0 conn(0x7f83c80b81c0 0x7f83c80cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83cd599700 1 -- 172.21.15.84:0/4251270503 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570531957 0 0) 0x7f83c80d1830 con 0x7f83c80ceda0 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83cd599700 1 -- 172.21.15.84:0/4251270503 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83b40035b0 con 0x7f83c80ceda0 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83ce59b700 1 -- 172.21.15.84:0/4251270503 >> v1:172.21.15.107:6789/0 conn(0x7f83c80b1a80 legacy=0x7f83c815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83ce59b700 1 -- 172.21.15.84:0/4251270503 >> v1:172.21.15.84:6789/0 conn(0x7f83c80ceda0 legacy=0x7f83c8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83ce59b700 1 -- 172.21.15.84:0/4251270503 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80d1270 con 0x7f83c80b81c0 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83ce59b700 1 --2- 172.21.15.84:0/4251270503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c80cf600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009840 tx=0x7f83c4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff05b021a0ca278e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83cd599700 1 -- 172.21.15.84:0/4251270503 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4011830 con 0x7f83c80b81c0 2023-07-06T21:42:03.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83cd599700 1 -- 172.21.15.84:0/4251270503 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c4011990 con 0x7f83c80b81c0 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.543+0000 7f83cd599700 1 -- 172.21.15.84:0/4251270503 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4011d20 con 0x7f83c80b81c0 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 -- 172.21.15.84:0/4251270503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 msgr2=0x7f83c80cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 --2- 172.21.15.84:0/4251270503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c80cf600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009840 tx=0x7f83c4009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 -- 172.21.15.84:0/4251270503 shutdown_connections 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 --2- 172.21.15.84:0/4251270503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c80cf600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 -- 172.21.15.84:0/4251270503 >> 172.21.15.84:0/4251270503 conn(0x7f83c806b2b0 msgr2=0x7f83c806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 -- 172.21.15.84:0/4251270503 shutdown_connections 2023-07-06T21:42:03.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.544+0000 7f83d07ff700 1 -- 172.21.15.84:0/4251270503 wait complete. 2023-07-06T21:42:03.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.546+0000 7f83d07ff700 1 Processor -- start 2023-07-06T21:42:03.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.546+0000 7f83d07ff700 1 -- start start 2023-07-06T21:42:03.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83d07ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 0x7f83c80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83d07ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c81f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83d07ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80ceda0 0x7f83c815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83d07ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80c8590 con 0x7f83c80b1a80 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83d07ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80ccc10 con 0x7f83c80b81c0 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83d07ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81f5f50 con 0x7f83c80ceda0 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 0x7f83c80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 0x7f83c80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54030/0 (socket says 172.21.15.84:54030) 2023-07-06T21:42:03.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 -- 172.21.15.84:0/3693907274 learned_addr learned my addr 172.21.15.84:0/3693907274 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 -- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80ceda0 msgr2=0x7f83c815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80ceda0 0x7f83c815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 -- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 msgr2=0x7f83c81f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83cdd9a700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c81f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c81f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83ce59b700 1 -- 172.21.15.84:0/3693907274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c40094a0 con 0x7f83c80b1a80 2023-07-06T21:42:03.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.547+0000 7f83cdd9a700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c81f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:03.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.548+0000 7f83ce59b700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 0x7f83c80cc6d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009fd0 tx=0x7f83c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:03.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.548+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40038a0 con 0x7f83c80b1a80 2023-07-06T21:42:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.548+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c401f070 con 0x7f83c80b1a80 2023-07-06T21:42:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.548+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4003c40 con 0x7f83c80b1a80 2023-07-06T21:42:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.548+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c81f53d0 con 0x7f83c80b1a80 2023-07-06T21:42:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.548+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c81f5990 con 0x7f83c80b1a80 2023-07-06T21:42:03.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.549+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c804e1d0 con 0x7f83c80b1a80 2023-07-06T21:42:03.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.550+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83c4026480 con 0x7f83c80b1a80 2023-07-06T21:42:03.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.551+0000 7f83beffd700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f83ac06ace0 0x7f83ac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.551+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f83c4021070 con 0x7f83c80b1a80 2023-07-06T21:42:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.551+0000 7f83cdd9a700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f83ac06ace0 0x7f83ac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.552+0000 7f83cdd9a700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f83ac06ace0 0x7f83ac06d190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f83c814c960 tx=0x7f83b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:03.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.555+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83c4055f00 con 0x7f83c80b1a80 2023-07-06T21:42:03.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.752+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f83c80b4f30 con 0x7f83c80b1a80 2023-07-06T21:42:03.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.756+0000 7f83beffd700 1 -- 172.21.15.84:0/3693907274 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83c40150c0 con 0x7f83c80b1a80 2023-07-06T21:42:03.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f83ac06ace0 msgr2=0x7f83ac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f83ac06ace0 0x7f83ac06d190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f83c814c960 tx=0x7f83b8009040 comp rx=0 tx=0).stop 2023-07-06T21:42:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 msgr2=0x7f83c80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 0x7f83c80cc6d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f83c4009fd0 tx=0x7f83c4004750 comp rx=0 tx=0).stop 2023-07-06T21:42:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 shutdown_connections 2023-07-06T21:42:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f83ac06ace0 0x7f83ac06d190 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80ceda0 0x7f83c815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83c80b1a80 0x7f83c80cc6d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 --2- 172.21.15.84:0/3693907274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c80b81c0 0x7f83c81f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 >> 172.21.15.84:0/3693907274 conn(0x7f83c806b2b0 msgr2=0x7f83c806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 shutdown_connections 2023-07-06T21:42:03.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:03.759+0000 7f83d07ff700 1 -- 172.21.15.84:0/3693907274 wait complete. 2023-07-06T21:42:03.779 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T21:42:03.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:42:03.879 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:42:03.985 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T21:42:03.986 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:03.987 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:04.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:42:04.031 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:42:04.031 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:04.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:42:04.077 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:04.077 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:04.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:42:04.120 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:42:04.121 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:04.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:42:04.166 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-07-06T21:42:04.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:04.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.358+0000 7fa57a3ff700 1 Processor -- start 2023-07-06T21:42:04.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.358+0000 7fa57a3ff700 1 -- start start 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.359+0000 7fa57a3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa574154910 0x7fa574070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa57a3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa574070230 con 0x7fa5740c9410 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa57a3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57415b2a0 con 0x7fa57415a7e0 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa57a3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa57415b580 con 0x7fa574154910 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa57899c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa5740c9410 0x7fa5740c3600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45132/0 (socket says 172.21.15.84:45132) 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa57899c700 1 -- 172.21.15.84:0/430282337 learned_addr learned my addr 172.21.15.84:0/430282337 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa573fff700 1 --2- 172.21.15.84:0/430282337 >> v2:172.21.15.107:3300/0 conn(0x7fa574154910 0x7fa574070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594806529 0 0) 0x7fa574070230 con 0x7fa5740c9410 2023-07-06T21:42:04.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa55c0035b0 con 0x7fa5740c9410 2023-07-06T21:42:04.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745450862 0 0) 0x7fa57415b2a0 con 0x7fa57415a7e0 2023-07-06T21:42:04.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa574070230 con 0x7fa57415a7e0 2023-07-06T21:42:04.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa573fff700 1 -- 172.21.15.84:0/430282337 >> v1:172.21.15.107:6789/0 conn(0x7fa57415a7e0 legacy=0x7fa5740c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa573fff700 1 -- 172.21.15.84:0/430282337 >> v1:172.21.15.84:6789/0 conn(0x7fa5740c9410 legacy=0x7fa5740c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.360+0000 7fa573fff700 1 -- 172.21.15.84:0/430282337 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa574070480 con 0x7fa574154910 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.361+0000 7fa573fff700 1 --2- 172.21.15.84:0/430282337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574154910 0x7fa574070c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa564009840 tx=0x7fa564009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5d566be034f10d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.361+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011830 con 0x7fa574154910 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.361+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564011990 con 0x7fa574154910 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.361+0000 7fa572ffd700 1 -- 172.21.15.84:0/430282337 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564011d20 con 0x7fa574154910 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.361+0000 7fa57a3ff700 1 -- 172.21.15.84:0/430282337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574154910 msgr2=0x7fa574070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.361+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/430282337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574154910 0x7fa574070c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa564009840 tx=0x7fa564009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.362+0000 7fa57a3ff700 1 -- 172.21.15.84:0/430282337 shutdown_connections 2023-07-06T21:42:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.362+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/430282337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa574154910 0x7fa574070c80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.362+0000 7fa57a3ff700 1 -- 172.21.15.84:0/430282337 >> 172.21.15.84:0/430282337 conn(0x7fa5740cf3d0 msgr2=0x7fa5740cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.362+0000 7fa57a3ff700 1 -- 172.21.15.84:0/430282337 shutdown_connections 2023-07-06T21:42:04.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.362+0000 7fa57a3ff700 1 -- 172.21.15.84:0/430282337 wait complete. 2023-07-06T21:42:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.363+0000 7fa57a3ff700 1 Processor -- start 2023-07-06T21:42:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.363+0000 7fa57a3ff700 1 -- start start 2023-07-06T21:42:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.364+0000 7fa57a3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5740c9410 0x7fa57414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.364+0000 7fa57a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa574154910 0x7fa5740cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.364+0000 7fa57a3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 0x7fa5740cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.364+0000 7fa57a3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574151970 con 0x7fa57415a7e0 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.364+0000 7fa57a3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57414c310 con 0x7fa5740c9410 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.364+0000 7fa57a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa574151d90 con 0x7fa574154910 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 0x7fa5740cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 0x7fa5740cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54044/0 (socket says 172.21.15.84:54044) 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 -- 172.21.15.84:0/2278153048 learned_addr learned my addr 172.21.15.84:0/2278153048 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa573fff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5740c9410 0x7fa57414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa5737fe700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa574154910 0x7fa5740cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 -- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa574154910 msgr2=0x7fa5740cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa574154910 0x7fa5740cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 -- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5740c9410 msgr2=0x7fa57414bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5740c9410 0x7fa57414bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 -- 172.21.15.84:0/2278153048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640094a0 con 0x7fa57415a7e0 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.365+0000 7fa57899c700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 0x7fa5740cca60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00cb40 tx=0x7fa56c00ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:04.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.366+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00a980 con 0x7fa57415a7e0 2023-07-06T21:42:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.366+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56c00aae0 con 0x7fa57415a7e0 2023-07-06T21:42:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.366+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa574150cf0 con 0x7fa57415a7e0 2023-07-06T21:42:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.366+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56c00ae90 con 0x7fa57415a7e0 2023-07-06T21:42:04.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.366+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5741512b0 con 0x7fa57415a7e0 2023-07-06T21:42:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.367+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa574151ab0 con 0x7fa57415a7e0 2023-07-06T21:42:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.368+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa56c01b3b0 con 0x7fa57415a7e0 2023-07-06T21:42:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.368+0000 7fa570ff9700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa56006ace0 0x7fa56006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.369+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa56c088da0 con 0x7fa57415a7e0 2023-07-06T21:42:04.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.369+0000 7fa573fff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa56006ace0 0x7fa56006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.369+0000 7fa573fff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa56006ace0 0x7fa56006d190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa564009fd0 tx=0x7fa56401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:04.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.373+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa574151ab0 con 0x7fa57415a7e0 2023-07-06T21:42:04.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.609+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa574151ab0 con 0x7fa57415a7e0 2023-07-06T21:42:04.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.610+0000 7fa570ff9700 1 -- 172.21.15.84:0/2278153048 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fa574151ab0 con 0x7fa57415a7e0 2023-07-06T21:42:04.613 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:04.613 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3954135713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":3954135713},{"type":"v1","addr":"172.21.15.107:6835","nonce":3954135713}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5212,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/578414291","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":578414291},{"type":"v1","addr":"172.21.15.107:6834","nonce":578414291}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5313,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/2567879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":2567879791},{"type":"v1","addr":"172.21.15.84:6836","nonce":2567879791}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5752,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/258968840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":258968840},{"type":"v1","addr":"172.21.15.84:6837","nonce":258968840}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-07-06T21:42:04.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa56006ace0 msgr2=0x7fa56006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa56006ace0 0x7fa56006d190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa564009fd0 tx=0x7fa56401c040 comp rx=0 tx=0).stop 2023-07-06T21:42:04.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 msgr2=0x7fa5740cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 0x7fa5740cca60 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00cb40 tx=0x7fa56c00ce50 comp rx=0 tx=0).stop 2023-07-06T21:42:04.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 shutdown_connections 2023-07-06T21:42:04.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa56006ace0 0x7fa56006d190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa57415a7e0 0x7fa5740cca60 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa5740c9410 0x7fa57414bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 --2- 172.21.15.84:0/2278153048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa574154910 0x7fa5740cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.612+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 >> 172.21.15.84:0/2278153048 conn(0x7fa5740cf3d0 msgr2=0x7fa5740cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.613+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 shutdown_connections 2023-07-06T21:42:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.613+0000 7fa57a3ff700 1 -- 172.21.15.84:0/2278153048 wait complete. 2023-07-06T21:42:04.617 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 18 2023-07-06T21:42:04.633 INFO:tasks.ceph.mds.a:Restarting daemon 2023-07-06T21:42:04.633 INFO:tasks.ceph.mds.a:Stopping old one... 2023-07-06T21:42:04.633 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T21:42:04.634 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:42:04.634 INFO:tasks.ceph.mds.c:Restarting daemon 2023-07-06T21:42:04.634 INFO:tasks.ceph.mds.c:Stopping old one... 2023-07-06T21:42:04.634 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T21:42:04.634 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:42:04.635 INFO:tasks.ceph.mds.b:Restarting daemon 2023-07-06T21:42:04.635 INFO:tasks.ceph.mds.b:Stopping old one... 2023-07-06T21:42:04.635 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T21:42:04.635 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:42:04.635 INFO:tasks.ceph.mds.d:Restarting daemon 2023-07-06T21:42:04.636 INFO:tasks.ceph.mds.d:Stopping old one... 2023-07-06T21:42:04.636 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T21:42:04.636 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:42:04.648 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T21:42:04.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-07-06T21:42:04.649 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T21:42:04.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-07-06T21:42:04.651 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T21:42:04.652 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-07-06T21:42:04.652 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T21:42:04.653 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-07-06T21:42:04.693 INFO:tasks.ceph.mds.a:Started 2023-07-06T21:42:04.694 DEBUG:teuthology.parallel:result is None 2023-07-06T21:42:04.695 INFO:tasks.ceph.mds.c:Started 2023-07-06T21:42:04.695 DEBUG:teuthology.parallel:result is None 2023-07-06T21:42:04.696 INFO:tasks.ceph.mds.d:Started 2023-07-06T21:42:04.696 DEBUG:teuthology.parallel:result is None 2023-07-06T21:42:04.698 INFO:tasks.ceph.mds.b:Started 2023-07-06T21:42:04.699 DEBUG:teuthology.parallel:result is None 2023-07-06T21:42:04.699 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-07-06T21:42:04.839 INFO:tasks.ceph.mds.c.smithi084.stdout:starting mds.c at 2023-07-06T21:42:04.841 INFO:tasks.ceph.mds.a.smithi084.stdout:starting mds.a at 2023-07-06T21:42:04.846 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-06T21:42:04.847 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2023-07-06T21:42:04.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.907+0000 7f6a71fff700 1 Processor -- start 2023-07-06T21:42:04.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.907+0000 7f6a71fff700 1 -- start start 2023-07-06T21:42:04.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.908+0000 7f6a71fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a71fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a6c0d3710 0x7f6a6c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a71fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0cfee0 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a71fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c0d0a80 con 0x7f6a6c0d3710 2023-07-06T21:42:04.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a71fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b8790 con 0x7f6a6c0b83c0 2023-07-06T21:42:04.911 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a6c0d3710 0x7f6a6c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6a6c0d3710 0x7f6a6c0cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54080/0 (socket says 172.21.15.84:54080) 2023-07-06T21:42:04.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 -- 172.21.15.84:0/2248602425 learned_addr learned my addr 172.21.15.84:0/2248602425 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:04.912 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6b7fe700 1 --2- 172.21.15.84:0/2248602425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 -- 172.21.15.84:0/2248602425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 msgr2=0x7f6a6c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:04.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 --2- 172.21.15.84:0/2248602425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.913 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 -- 172.21.15.84:0/2248602425 >> v1:172.21.15.107:6789/0 conn(0x7f6a6c0cfa90 legacy=0x7f6a6c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.909+0000 7f6a6affd700 1 -- 172.21.15.84:0/2248602425 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0b4e70 con 0x7f6a6c0d3710 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a6affd700 1 --2- 172.21.15.84:0/2248602425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c0cf550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6a600098f0 tx=0x7f6a60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daccdcbe1453dd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a6a7fc700 1 -- 172.21.15.84:0/2248602425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000b8a0 con 0x7f6a6c0d3710 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a6a7fc700 1 -- 172.21.15.84:0/2248602425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6000ba00 con 0x7f6a6c0d3710 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a6a7fc700 1 -- 172.21.15.84:0/2248602425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000bd90 con 0x7f6a6c0d3710 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 -- 172.21.15.84:0/2248602425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 msgr2=0x7f6a6c0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.914 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 --2- 172.21.15.84:0/2248602425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c0cf550 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6a600098f0 tx=0x7f6a60009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:04.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 -- 172.21.15.84:0/2248602425 shutdown_connections 2023-07-06T21:42:04.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 --2- 172.21.15.84:0/2248602425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 --2- 172.21.15.84:0/2248602425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c0cf550 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 -- 172.21.15.84:0/2248602425 >> 172.21.15.84:0/2248602425 conn(0x7f6a6c06b350 msgr2=0x7f6a6c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:04.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 -- 172.21.15.84:0/2248602425 shutdown_connections 2023-07-06T21:42:04.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.910+0000 7f6a71fff700 1 -- 172.21.15.84:0/2248602425 wait complete. 2023-07-06T21:42:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.912+0000 7f6a71fff700 1 Processor -- start 2023-07-06T21:42:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.912+0000 7f6a71fff700 1 -- start start 2023-07-06T21:42:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a71fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.917 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a71fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a6c0cfa90 0x7f6a6c159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a71fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a71fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1f60e0 con 0x7f6a6c0d3710 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a71fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c1f54f0 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a71fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c14de50 con 0x7f6a6c0b83c0 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a6affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a6c0cfa90 0x7f6a6c159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c14d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:50304/0 (socket says 172.21.15.84:50304) 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.913+0000 7f6a6b7fe700 1 -- 172.21.15.84:0/541560005 learned_addr learned my addr 172.21.15.84:0/541560005 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a6affd700 1 -- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 msgr2=0x7f6a6c14d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a6affd700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c14d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a6affd700 1 -- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 msgr2=0x7f6a6c159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a6affd700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c159ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a6affd700 1 -- 172.21.15.84:0/541560005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a60009550 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a6affd700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a6c0cfa90 0x7f6a6c159580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6a60016000 tx=0x7f6a60004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:04.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60004510 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6001e070 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c1f57b0 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.914+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a600038e0 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.915+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c1f5d70 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.915+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c1f6250 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.916+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6a60003a40 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.917+0000 7f6a53fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a5806ace0 0x7f6a5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:04.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.917+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f6a60020070 con 0x7f6a6c0cfa90 2023-07-06T21:42:04.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.918+0000 7f6a6b7fe700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a5806ace0 0x7f6a5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:04.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.918+0000 7f6a6b7fe700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a5806ace0 0x7f6a5806d190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007af0 tx=0x7f6a5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:04.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:04.923+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6a600557b0 con 0x7f6a6c0cfa90 2023-07-06T21:42:05.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:05.140+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6a6c160530 con 0x7f6a6c0cfa90 2023-07-06T21:42:06.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.065+0000 7f6a53fff700 1 -- 172.21.15.84:0/541560005 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6a60058dd0 con 0x7f6a6c0cfa90 2023-07-06T21:42:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.068+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a5806ace0 msgr2=0x7f6a5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.068+0000 7f6a71fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a5806ace0 0x7f6a5806d190 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c007af0 tx=0x7f6a5c008040 comp rx=0 tx=0).stop 2023-07-06T21:42:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.068+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a6c0cfa90 msgr2=0x7f6a6c159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.068+0000 7f6a71fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a6c0cfa90 0x7f6a6c159580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6a60016000 tx=0x7f6a60004750 comp rx=0 tx=0).stop 2023-07-06T21:42:06.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 shutdown_connections 2023-07-06T21:42:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6a5806ace0 0x7f6a5806d190 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6a6c0d3710 0x7f6a6c159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a6c0b83c0 0x7f6a6c14d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 --2- 172.21.15.84:0/541560005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a6c0cfa90 0x7f6a6c159580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 >> 172.21.15.84:0/541560005 conn(0x7f6a6c06b350 msgr2=0x7f6a6c0b6bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:06.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 shutdown_connections 2023-07-06T21:42:06.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.069+0000 7f6a71fff700 1 -- 172.21.15.84:0/541560005 wait complete. 2023-07-06T21:42:06.073 INFO:teuthology.orchestra.run.smithi084.stderr: removed all blocklist entries 2023-07-06T21:42:06.092 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-07-06T21:42:06.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.287+0000 7f371c1ff700 1 Processor -- start 2023-07-06T21:42:06.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.287+0000 7f371c1ff700 1 -- start start 2023-07-06T21:42:06.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.288+0000 7f371c1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f37140d2690 0x7f37140cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.288+0000 7f371c1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37140d1460 con 0x7f37140b1fd0 2023-07-06T21:42:06.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f371c1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37140d1740 con 0x7f37140b59c0 2023-07-06T21:42:06.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f371c1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37140d2530 con 0x7f37140d2690 2023-07-06T21:42:06.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3719f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f37140b1fd0 0x7f37140cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45156/0 (socket says 172.21.15.84:45156) 2023-07-06T21:42:06.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3719f9b700 1 -- 172.21.15.84:0/2520064786 learned_addr learned my addr 172.21.15.84:0/2520064786 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:06.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f371979a700 1 --2- 172.21.15.84:0/2520064786 >> v2:172.21.15.107:3300/0 conn(0x7f37140d2690 0x7f37140cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973600700 0 0) 0x7f37140d1460 con 0x7f37140b1fd0 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37000035b0 con 0x7f37140b1fd0 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421719486 0 0) 0x7f37000035b0 con 0x7f37140b1fd0 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 >> v2:172.21.15.107:3300/0 conn(0x7f37140d2690 msgr2=0x7f37140cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 --2- 172.21.15.84:0/2520064786 >> v2:172.21.15.107:3300/0 conn(0x7f37140d2690 0x7f37140cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 >> v1:172.21.15.107:6789/0 conn(0x7f37140b59c0 legacy=0x7f371415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37140cd260 con 0x7f37140b1fd0 2023-07-06T21:42:06.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f3710001670 con 0x7f37140b1fd0 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.289+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 >> v1:172.21.15.84:6789/0 conn(0x7f37140b1fd0 legacy=0x7f37140cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3718f99700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3700003a50 0x7f3700003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3718f99700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3700004850 0x7f3700004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3718f99700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700005250 0x7f370000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140d2530 con 0x7f3700005250 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370000c160 con 0x7f3700003a50 2023-07-06T21:42:06.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370000c320 con 0x7f3700004850 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3700003a50 0x7f3700003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f371a79c700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700005250 0x7f370000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f371979a700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3700004850 0x7f3700004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3719f9b700 1 -- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3700004850 msgr2=0x7f3700004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3700004850 0x7f3700004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3719f9b700 1 -- 172.21.15.84:0/2520064786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700005250 msgr2=0x7f370000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700005250 0x7f370000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f371a79c700 1 -- 172.21.15.84:0/2520064786 reap_dead start 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.290+0000 7f3719f9b700 1 -- 172.21.15.84:0/2520064786 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37000035b0 con 0x7f3700003a50 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3700003a50 0x7f3700003e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f371000c160 tx=0x7f3700005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30c1fa4d5b5077c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710007b70 con 0x7f3700003a50 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371000cd20 con 0x7f3700003a50 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f3718f99700 1 -- 172.21.15.84:0/2520064786 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710005ba0 con 0x7f3700003a50 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f371c1ff700 1 -- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3700003a50 msgr2=0x7f3700003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3700003a50 0x7f3700003e60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f371000c160 tx=0x7f3700005880 comp rx=0 tx=0).stop 2023-07-06T21:42:06.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f371c1ff700 1 -- 172.21.15.84:0/2520064786 shutdown_connections 2023-07-06T21:42:06.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2520064786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3700003a50 0x7f3700003e60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.291+0000 7f371c1ff700 1 -- 172.21.15.84:0/2520064786 >> 172.21.15.84:0/2520064786 conn(0x7f371406b340 msgr2=0x7f371406b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:06.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.292+0000 7f371c1ff700 1 -- 172.21.15.84:0/2520064786 shutdown_connections 2023-07-06T21:42:06.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.292+0000 7f371c1ff700 1 -- 172.21.15.84:0/2520064786 wait complete. 2023-07-06T21:42:06.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.293+0000 7f371c1ff700 1 Processor -- start 2023-07-06T21:42:06.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.293+0000 7f371c1ff700 1 -- start start 2023-07-06T21:42:06.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.294+0000 7f371c1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 0x7f3714151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.294+0000 7f371c1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3714155920 0x7f37140ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.294+0000 7f371c1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37140ca950 0x7f37140cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.294+0000 7f371c1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140cb0b0 con 0x7f3700003a50 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.294+0000 7f371c1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37141f4fc0 con 0x7f37140ca950 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.294+0000 7f371c1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140cadd0 con 0x7f3714155920 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 0x7f3714151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 0x7f3714151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54112/0 (socket says 172.21.15.84:54112) 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 -- 172.21.15.84:0/2719205893 learned_addr learned my addr 172.21.15.84:0/2719205893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f371a79c700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37140ca950 0x7f37140cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f371979a700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3714155920 0x7f37140ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 -- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3714155920 msgr2=0x7f37140ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3714155920 0x7f37140ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 -- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37140ca950 msgr2=0x7f37140cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37140ca950 0x7f37140cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 -- 172.21.15.84:0/2719205893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37141f5210 con 0x7f3700003a50 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f3719f9b700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 0x7f3714151c20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f370000c2f0 tx=0x7f3710001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.295+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710004c80 con 0x7f3700003a50 2023-07-06T21:42:06.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.296+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3710004e00 con 0x7f3700003a50 2023-07-06T21:42:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.296+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3710007310 con 0x7f3700003a50 2023-07-06T21:42:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.296+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37141f54f0 con 0x7f3700003a50 2023-07-06T21:42:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.296+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3714152160 con 0x7f3700003a50 2023-07-06T21:42:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.297+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37141e6d50 con 0x7f3700003a50 2023-07-06T21:42:06.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.298+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3710007470 con 0x7f3700003a50 2023-07-06T21:42:06.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.298+0000 7f370a7fc700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f806ace0 0x7f36f806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.299+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3710089940 con 0x7f3700003a50 2023-07-06T21:42:06.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.299+0000 7f371979a700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f806ace0 0x7f36f806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.299+0000 7f371979a700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f806ace0 0x7f36f806d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f371404fa50 tx=0x7f3704009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:06.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.303+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3710054db0 con 0x7f3700003a50 2023-07-06T21:42:06.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.576+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f37140b82c0 con 0x7f3700003a50 2023-07-06T21:42:06.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.577+0000 7f370a7fc700 1 -- 172.21.15.84:0/2719205893 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f3710059e60 con 0x7f3700003a50 2023-07-06T21:42:06.579 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:06.579 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:42:06.579 INFO:teuthology.orchestra.run.smithi084.stdout: "auth_dump": [ 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.a", 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdk+/m1ERAAVuVl/rTq2tc0Dhv83fjupA==", 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:42:06.580 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.b", 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdkRtxvKRAAIt8IaHnqNpNdKNcNdnoN7g==", 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.581 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.c", 2023-07-06T21:42:06.582 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdkpgiGHRAA+dmQ4tCjJggxC0UEAx4E8A==", 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mds.d", 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdkdtB1NRAAZPgJEifPgIZTnZI5Sr0ZIw==", 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:42:06.584 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.0", 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBRNKdko5ENMxAAkg9Bb14zud5gryZG5cNFmA==", 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.585 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.1", 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBUNKdkIAyGOhAAGm6K8mPZUi+KIOhlHHcf/A==", 2023-07-06T21:42:06.586 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.2", 2023-07-06T21:42:06.587 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBYNKdkPYfnBBAAQiGXT+pTVZ/rM8G100/dGQ==", 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.588 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.3", 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBbNKdklfeTCxAAEMCI574RRSmAnzz2hU00Jg==", 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.4", 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBhNKdkluR8IxAAd2WQtPkTN09bIdqHrbRBCg==", 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.5", 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBkNKdkH7wwKRAAyXE0BPwP8/koUUWCcCA1dQ==", 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.591 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.592 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.592 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.6", 2023-07-06T21:42:06.592 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBnNKdk363RMBAArv8mMHQtqZRUf3AN2PqwTQ==", 2023-07-06T21:42:06.592 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "osd.7", 2023-07-06T21:42:06.593 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBqNKdkTpaXOBAACmQ850h7EOrJ8sl0ODNYmA==", 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow profile osd", 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile osd", 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.594 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.0", 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdkqmQnBBAALedUX3/ZDQ9mAhOGEH6QrA==", 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow r", 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rw pool=cephfs_data" 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.1", 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdkLt+uExAAzCoF/xhx1jkry5bkwObTEQ==", 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:42:06.596 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.2", 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdk0bfQCxAA+4nQgIo/iEDM930/C0JV1w==", 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.597 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.3", 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBONKdkodhbGxAA8n9L0VrpLjBlrDyEi4Sztg==", 2023-07-06T21:42:06.598 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow", 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow r", 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow rw", 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow rwx" 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.599 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "client.admin", 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBMNKdklNf3ExAAg1unuhVMRgLZ3q8jZCwR0g==", 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr": "allow *", 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow *", 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.x", 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBNNKdk8JijCRAAGnCGHPatWKPv0GPKCjXNtg==", 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-07-06T21:42:06.601 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: "entity": "mgr.y", 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: "key": "AQBMNKdkfcUzOxAA9BUPVthEZwBTmBTqdmDopg==", 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": { 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": "allow *", 2023-07-06T21:42:06.602 INFO:teuthology.orchestra.run.smithi084.stdout: "mon": "allow profile mgr", 2023-07-06T21:42:06.603 INFO:teuthology.orchestra.run.smithi084.stdout: "osd": "allow *" 2023-07-06T21:42:06.603 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.603 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:06.603 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2023-07-06T21:42:06.603 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:42:06.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f806ace0 msgr2=0x7f36f806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f806ace0 0x7f36f806d190 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f371404fa50 tx=0x7f3704009210 comp rx=0 tx=0).stop 2023-07-06T21:42:06.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 msgr2=0x7f3714151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 0x7f3714151c20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f370000c2f0 tx=0x7f3710001930 comp rx=0 tx=0).stop 2023-07-06T21:42:06.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 shutdown_connections 2023-07-06T21:42:06.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f36f806ace0 0x7f36f806d190 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37140ca950 0x7f37140cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3700003a50 0x7f3714151c20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 --2- 172.21.15.84:0/2719205893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3714155920 0x7f37140ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 >> 172.21.15.84:0/2719205893 conn(0x7f371406b340 msgr2=0x7f37140732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:06.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 shutdown_connections 2023-07-06T21:42:06.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.579+0000 7f371c1ff700 1 -- 172.21.15.84:0/2719205893 wait complete. 2023-07-06T21:42:06.607 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:06.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.797+0000 7f88747ff700 1 Processor -- start 2023-07-06T21:42:06.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.798+0000 7f88747ff700 1 -- start start 2023-07-06T21:42:06.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f88747ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f886c0d1580 0x7f886c166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f88747ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0d0990 con 0x7f886c0d3580 2023-07-06T21:42:06.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f88747ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0cd9d0 con 0x7f886c0b8530 2023-07-06T21:42:06.801 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f88747ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886c0d3af0 con 0x7f886c0d1580 2023-07-06T21:42:06.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f8872d9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f886c0d1580 0x7f886c166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f8871d9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f886c0d3580 0x7f886c0d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45166/0 (socket says 172.21.15.84:45166) 2023-07-06T21:42:06.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.799+0000 7f8871d9a700 1 -- 172.21.15.84:0/2817925580 learned_addr learned my addr 172.21.15.84:0/2817925580 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:06.802 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8872d9c700 1 -- 172.21.15.84:0/2817925580 >> v1:172.21.15.107:6789/0 conn(0x7f886c0b8530 legacy=0x7f886c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8872d9c700 1 -- 172.21.15.84:0/2817925580 >> v1:172.21.15.84:6789/0 conn(0x7f886c0d3580 legacy=0x7f886c0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8872d9c700 1 -- 172.21.15.84:0/2817925580 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c0cd410 con 0x7f886c0d1580 2023-07-06T21:42:06.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8872d9c700 1 --2- 172.21.15.84:0/2817925580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0d1580 0x7f886c166ff0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8864009d90 tx=0x7f886400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bf6019197f4994 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:06.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8871599700 1 -- 172.21.15.84:0/2817925580 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886400e070 con 0x7f886c0d1580 2023-07-06T21:42:06.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8871599700 1 -- 172.21.15.84:0/2817925580 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400bbd0 con 0x7f886c0d1580 2023-07-06T21:42:06.803 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.800+0000 7f8871599700 1 -- 172.21.15.84:0/2817925580 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864012430 con 0x7f886c0d1580 2023-07-06T21:42:06.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 -- 172.21.15.84:0/2817925580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0d1580 msgr2=0x7f886c166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 --2- 172.21.15.84:0/2817925580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0d1580 0x7f886c166ff0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8864009d90 tx=0x7f886400b330 comp rx=0 tx=0).stop 2023-07-06T21:42:06.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 -- 172.21.15.84:0/2817925580 shutdown_connections 2023-07-06T21:42:06.804 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 --2- 172.21.15.84:0/2817925580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0d1580 0x7f886c166ff0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 -- 172.21.15.84:0/2817925580 >> 172.21.15.84:0/2817925580 conn(0x7f886c0715c0 msgr2=0x7f886c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:06.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 -- 172.21.15.84:0/2817925580 shutdown_connections 2023-07-06T21:42:06.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.801+0000 7f88747ff700 1 -- 172.21.15.84:0/2817925580 wait complete. 2023-07-06T21:42:06.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.803+0000 7f88747ff700 1 Processor -- start 2023-07-06T21:42:06.805 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.803+0000 7f88747ff700 1 -- start start 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.804+0000 7f88747ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 0x7f886c1551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.804+0000 7f88747ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f886c0d1580 0x7f886c157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.804+0000 7f88747ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f886c0d3580 0x7f886c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f88747ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c156f60 con 0x7f886c0b8530 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f88747ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0c9f20 con 0x7f886c0d3580 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f88747ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c155720 con 0x7f886c0d1580 2023-07-06T21:42:06.807 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 0x7f886c1551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 0x7f886c1551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54142/0 (socket says 172.21.15.84:54142) 2023-07-06T21:42:06.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 -- 172.21.15.84:0/3498777079 learned_addr learned my addr 172.21.15.84:0/3498777079 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:06.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f8872d9c700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f886c0d3580 0x7f886c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f8871d9a700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f886c0d1580 0x7f886c157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 -- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f886c0d1580 msgr2=0x7f886c157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f886c0d1580 0x7f886c157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 -- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f886c0d3580 msgr2=0x7f886c157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:06.809 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f886c0d3580 0x7f886c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 -- 172.21.15.84:0/3498777079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8864009a70 con 0x7f886c0b8530 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.805+0000 7f887259b700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 0x7f886c1551e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f886800c4d0 tx=0x7f886800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.806+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88680052d0 con 0x7f886c0b8530 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.806+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8868005430 con 0x7f886c0b8530 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.806+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88680038f0 con 0x7f886c0b8530 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.806+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c1566c0 con 0x7f886c0b8530 2023-07-06T21:42:06.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.806+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c156c80 con 0x7f886c0b8530 2023-07-06T21:42:06.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.807+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c0c9cd0 con 0x7f886c0b8530 2023-07-06T21:42:06.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.808+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8868003a50 con 0x7f886c0b8530 2023-07-06T21:42:06.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.809+0000 7f8862ffd700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f885006ace0 0x7f885006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:06.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.809+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f88680886e0 con 0x7f886c0b8530 2023-07-06T21:42:06.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.809+0000 7f8871d9a700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f885006ace0 0x7f885006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:06.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.809+0000 7f8871d9a700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f885006ace0 0x7f885006d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f886c158e50 tx=0x7f885c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:06.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:06.813+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8868053a40 con 0x7f886c0b8530 2023-07-06T21:42:07.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.011+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f886c0d0090 con 0x7f886c0b8530 2023-07-06T21:42:07.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.012+0000 7f8862ffd700 1 -- 172.21.15.84:0/3498777079 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+14639 (secure 0 0 0) 0x7f8868057060 con 0x7f886c0b8530 2023-07-06T21:42:07.014 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:07.014 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":49,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:06.051683+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:07.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f885006ace0 msgr2=0x7f885006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f885006ace0 0x7f885006d190 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f886c158e50 tx=0x7f885c009040 comp rx=0 tx=0).stop 2023-07-06T21:42:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 msgr2=0x7f886c1551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 0x7f886c1551e0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f886800c4d0 tx=0x7f886800c890 comp rx=0 tx=0).stop 2023-07-06T21:42:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 shutdown_connections 2023-07-06T21:42:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f885006ace0 0x7f885006d190 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f886c0d3580 0x7f886c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f886c0b8530 0x7f886c1551e0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 --2- 172.21.15.84:0/3498777079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f886c0d1580 0x7f886c157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.014+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 >> 172.21.15.84:0/3498777079 conn(0x7f886c0715c0 msgr2=0x7f886c15e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:07.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.015+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 shutdown_connections 2023-07-06T21:42:07.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.015+0000 7f88747ff700 1 -- 172.21.15.84:0/3498777079 wait complete. 2023-07-06T21:42:07.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:07.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:07.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.233+0000 7f2082fff700 1 Processor -- start 2023-07-06T21:42:07.235 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.233+0000 7f2082fff700 1 -- start start 2023-07-06T21:42:07.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f2082fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f207c0cfd10 0x7f207c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f2082fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f207c0b1a80 0x7f207c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.236 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f2082fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f207c0d1830 con 0x7f207c0b81c0 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f2082fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0cf330 con 0x7f207c0b1a80 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f2082fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207c0d1270 con 0x7f207c0cfd10 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f208159c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f207c0b1a80 0x7f207c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f208159c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f207c0b1a80 0x7f207c0d28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54148/0 (socket says 172.21.15.84:54148) 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f208159c700 1 -- 172.21.15.84:0/156960779 learned_addr learned my addr 172.21.15.84:0/156960779 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f2080d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f207c0b81c0 0x7f207c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45168/0 (socket says 172.21.15.84:45168) 2023-07-06T21:42:07.237 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.234+0000 7f207bfff700 1 --2- 172.21.15.84:0/156960779 >> v2:172.21.15.107:3300/0 conn(0x7f207c0cfd10 0x7f207c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f208159c700 1 -- 172.21.15.84:0/156960779 >> v2:172.21.15.107:3300/0 conn(0x7f207c0cfd10 msgr2=0x7f207c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f208159c700 1 --2- 172.21.15.84:0/156960779 >> v2:172.21.15.107:3300/0 conn(0x7f207c0cfd10 0x7f207c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f208159c700 1 -- 172.21.15.84:0/156960779 >> v1:172.21.15.84:6789/0 conn(0x7f207c0b81c0 legacy=0x7f207c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f208159c700 1 -- 172.21.15.84:0/156960779 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c0d1550 con 0x7f207c0b1a80 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f207b7fe700 1 -- 172.21.15.84:0/156960779 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216977423 0 0) 0x7f207c0d1830 con 0x7f207c0b81c0 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f208159c700 1 --2- 172.21.15.84:0/156960779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0b1a80 0x7f207c0d28c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f207c0d0770 tx=0x7f207400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f7ab5ef44c9581 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:07.239 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f207b7fe700 1 -- 172.21.15.84:0/156960779 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207400e070 con 0x7f207c0b1a80 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f207b7fe700 1 -- 172.21.15.84:0/156960779 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207400b910 con 0x7f207c0b1a80 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.235+0000 7f207b7fe700 1 -- 172.21.15.84:0/156960779 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207400bcc0 con 0x7f207c0b1a80 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 -- 172.21.15.84:0/156960779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0b1a80 msgr2=0x7f207c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 --2- 172.21.15.84:0/156960779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0b1a80 0x7f207c0d28c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f207c0d0770 tx=0x7f207400b330 comp rx=0 tx=0).stop 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 -- 172.21.15.84:0/156960779 shutdown_connections 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 --2- 172.21.15.84:0/156960779 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0b1a80 0x7f207c0d28c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.240 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 --2- 172.21.15.84:0/156960779 >> v2:172.21.15.107:3300/0 conn(0x7f207c0cfd10 0x7f207c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 -- 172.21.15.84:0/156960779 >> 172.21.15.84:0/156960779 conn(0x7f207c06b2b0 msgr2=0x7f207c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:07.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 -- 172.21.15.84:0/156960779 shutdown_connections 2023-07-06T21:42:07.241 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.236+0000 7f2082fff700 1 -- 172.21.15.84:0/156960779 wait complete. 2023-07-06T21:42:07.242 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.238+0000 7f2082fff700 1 Processor -- start 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.238+0000 7f2082fff700 1 -- start start 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2082fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f207c0b1a80 0x7f207c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2082fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f207c0b81c0 0x7f207c0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2082fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 0x7f207c0b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2082fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0c7d10 con 0x7f207c0cfd10 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2082fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0c5f00 con 0x7f207c0b81c0 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2082fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f207c0c7090 con 0x7f207c0b1a80 2023-07-06T21:42:07.243 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 0x7f207c0b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 0x7f207c0b36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54152/0 (socket says 172.21.15.84:54152) 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 -- 172.21.15.84:0/2598995944 learned_addr learned my addr 172.21.15.84:0/2598995944 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f2080d9b700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f207c0b1a80 0x7f207c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f207bfff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f207c0b81c0 0x7f207c0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 -- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f207c0b1a80 msgr2=0x7f207c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f207c0b1a80 0x7f207c0c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.244 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 -- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f207c0b81c0 msgr2=0x7f207c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f207c0b81c0 0x7f207c0c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.239+0000 7f208159c700 1 -- 172.21.15.84:0/2598995944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074009b30 con 0x7f207c0cfd10 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.240+0000 7f208159c700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 0x7f207c0b36a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f207c06bed0 tx=0x7f2074004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.240+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20740099b0 con 0x7f207c0cfd10 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.240+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20740092e0 con 0x7f207c0cfd10 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.240+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207c0c7370 con 0x7f207c0cfd10 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.240+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207401c660 con 0x7f207c0cfd10 2023-07-06T21:42:07.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.240+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207c0c7930 con 0x7f207c0cfd10 2023-07-06T21:42:07.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.241+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f207c0c7e50 con 0x7f207c0cfd10 2023-07-06T21:42:07.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.242+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f207401c7c0 con 0x7f207c0cfd10 2023-07-06T21:42:07.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.243+0000 7f20797fa700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.244+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f207408ab40 con 0x7f207c0cfd10 2023-07-06T21:42:07.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.244+0000 7f2080d9b700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.244+0000 7f2080d9b700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2070000f60 tx=0x7f2070007be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:07.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.248+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f207c0c7e50 con 0x7f207c0cfd10 2023-07-06T21:42:07.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.443+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f207c0c7e50 con 0x7f207c0cfd10 2023-07-06T21:42:07.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.444+0000 7f20797fa700 1 -- 172.21.15.84:0/2598995944 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f207c0c7e50 con 0x7f207c0cfd10 2023-07-06T21:42:07.446 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 msgr2=0x7f206006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2070000f60 tx=0x7f2070007be0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 msgr2=0x7f207c0b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 0x7f207c0b36a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f207c06bed0 tx=0x7f2074004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 shutdown_connections 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f206006ace0 0x7f206006d190 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f207c0cfd10 0x7f207c0b36a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f207c0b1a80 0x7f207c0c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 --2- 172.21.15.84:0/2598995944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f207c0b81c0 0x7f207c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 >> 172.21.15.84:0/2598995944 conn(0x7f207c06b2b0 msgr2=0x7f207c0d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:07.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.446+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 shutdown_connections 2023-07-06T21:42:07.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.447+0000 7f2082fff700 1 -- 172.21.15.84:0/2598995944 wait complete. 2023-07-06T21:42:07.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:07.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.661+0000 7f3f411ff700 1 Processor -- start 2023-07-06T21:42:07.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.661+0000 7f3f411ff700 1 -- start start 2023-07-06T21:42:07.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f411ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f3f3c0cfd10 0x7f3f3c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f411ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0d1830 con 0x7f3f3c0b81c0 2023-07-06T21:42:07.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f411ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0cf330 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f411ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c0d1270 con 0x7f3f3c0cfd10 2023-07-06T21:42:07.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f3ad9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3f3c0b81c0 0x7f3f3c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45174/0 (socket says 172.21.15.84:45174) 2023-07-06T21:42:07.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f3ad9d700 1 -- 172.21.15.84:0/2064445088 learned_addr learned my addr 172.21.15.84:0/2064445088 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:07.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f33fff700 1 --2- 172.21.15.84:0/2064445088 >> v2:172.21.15.84:3300/0 conn(0x7f3f3c0cfd10 0x7f3f3c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f33fff700 1 -- 172.21.15.84:0/2064445088 >> v1:172.21.15.107:6789/0 conn(0x7f3f3c0b1a80 legacy=0x7f3f3c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f3a59c700 1 -- 172.21.15.84:0/2064445088 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305831487 0 0) 0x7f3f3c0d1830 con 0x7f3f3c0b81c0 2023-07-06T21:42:07.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f33fff700 1 -- 172.21.15.84:0/2064445088 >> v1:172.21.15.84:6789/0 conn(0x7f3f3c0b81c0 legacy=0x7f3f3c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.663+0000 7f3f33fff700 1 -- 172.21.15.84:0/2064445088 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0d1550 con 0x7f3f3c0cfd10 2023-07-06T21:42:07.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f33fff700 1 --2- 172.21.15.84:0/2064445088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cfd10 0x7f3f3c0d2380 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009840 tx=0x7f3f24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbaca1ea36e1226 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:07.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f3a59c700 1 -- 172.21.15.84:0/2064445088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24011830 con 0x7f3f3c0cfd10 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f3a59c700 1 -- 172.21.15.84:0/2064445088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f24011990 con 0x7f3f3c0cfd10 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f3a59c700 1 -- 172.21.15.84:0/2064445088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f24011d20 con 0x7f3f3c0cfd10 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 -- 172.21.15.84:0/2064445088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cfd10 msgr2=0x7f3f3c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 --2- 172.21.15.84:0/2064445088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cfd10 0x7f3f3c0d2380 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009840 tx=0x7f3f24009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 -- 172.21.15.84:0/2064445088 shutdown_connections 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 --2- 172.21.15.84:0/2064445088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0cfd10 0x7f3f3c0d2380 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 -- 172.21.15.84:0/2064445088 >> 172.21.15.84:0/2064445088 conn(0x7f3f3c06b2b0 msgr2=0x7f3f3c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:07.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 -- 172.21.15.84:0/2064445088 shutdown_connections 2023-07-06T21:42:07.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.664+0000 7f3f411ff700 1 -- 172.21.15.84:0/2064445088 wait complete. 2023-07-06T21:42:07.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.667+0000 7f3f411ff700 1 Processor -- start 2023-07-06T21:42:07.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.667+0000 7f3f411ff700 1 -- start start 2023-07-06T21:42:07.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f411ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 0x7f3f3c1f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f411ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0b81c0 0x7f3f3c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f411ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0cfd10 0x7f3f3c0b7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f411ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0b6c60 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f411ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0b6070 con 0x7f3f3c0b81c0 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f411ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0b62c0 con 0x7f3f3c0cfd10 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f3ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 0x7f3f3c1f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f3ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 0x7f3f3c1f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54182/0 (socket says 172.21.15.84:54182) 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.668+0000 7f3f3ad9d700 1 -- 172.21.15.84:0/993088721 learned_addr learned my addr 172.21.15.84:0/993088721 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f33fff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0b81c0 0x7f3f3c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3b59e700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0cfd10 0x7f3f3c0b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3ad9d700 1 -- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0cfd10 msgr2=0x7f3f3c0b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3ad9d700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0cfd10 0x7f3f3c0b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3ad9d700 1 -- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0b81c0 msgr2=0x7f3f3c156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3ad9d700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0b81c0 0x7f3f3c156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3ad9d700 1 -- 172.21.15.84:0/993088721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240094a0 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f33fff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0b81c0 0x7f3f3c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:07.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f3ad9d700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 0x7f3f3c1f52a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c520 tx=0x7f3f2c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:07.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c004c80 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c004e00 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c007310 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.669+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c0b65a0 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.670+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c1f5d20 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.670+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c0b6ed0 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.671+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3f2c007470 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.672+0000 7f3f337fe700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f1c06ace0 0x7f3f1c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:07.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.672+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3f2c088e70 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.672+0000 7f3f33fff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f1c06ace0 0x7f3f1c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:07.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.673+0000 7f3f33fff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f1c06ace0 0x7f3f1c06d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0d0100 tx=0x7f3f2401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:07.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.677+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3f2c054260 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.870+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3f3c04e1d0 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.871+0000 7f3f337fe700 1 -- 172.21.15.84:0/993088721 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v49) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3f2c057880 con 0x7f3f3c0b1a80 2023-07-06T21:42:07.873 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:07.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f1c06ace0 msgr2=0x7f3f1c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f1c06ace0 0x7f3f1c06d190 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0d0100 tx=0x7f3f2401c040 comp rx=0 tx=0).stop 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 msgr2=0x7f3f3c1f52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 0x7f3f3c1f52a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00c520 tx=0x7f3f2c00c8e0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 shutdown_connections 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3f1c06ace0 0x7f3f1c06d190 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0cfd10 0x7f3f3c0b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f3c0b1a80 0x7f3f3c1f52a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 --2- 172.21.15.84:0/993088721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0b81c0 0x7f3f3c156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:07.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 >> 172.21.15.84:0/993088721 conn(0x7f3f3c06b2b0 msgr2=0x7f3f3c0c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:07.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.873+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 shutdown_connections 2023-07-06T21:42:07.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:07.874+0000 7f3f411ff700 1 -- 172.21.15.84:0/993088721 wait complete. 2023-07-06T21:42:07.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-06T21:42:08.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.085+0000 7f5b18bff700 1 Processor -- start 2023-07-06T21:42:08.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.085+0000 7f5b18bff700 1 -- start start 2023-07-06T21:42:08.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b18bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b140b83c0 0x7f5b140cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b18bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5b140cfc60 0x7f5b140d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b18bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b140d0a80 con 0x7f5b140d3710 2023-07-06T21:42:08.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b18bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b140b8790 con 0x7f5b140b83c0 2023-07-06T21:42:08.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b18bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b140b4e70 con 0x7f5b140cfc60 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b1259c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b140b83c0 0x7f5b140cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.086+0000 7f5b1259c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b140b83c0 0x7f5b140cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54194/0 (socket says 172.21.15.84:54194) 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1259c700 1 -- 172.21.15.84:0/3242527669 learned_addr learned my addr 172.21.15.84:0/3242527669 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b12d9d700 1 --2- 172.21.15.84:0/3242527669 >> v2:172.21.15.107:3300/0 conn(0x7f5b140cfc60 0x7f5b140d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1259c700 1 -- 172.21.15.84:0/3242527669 >> v2:172.21.15.107:3300/0 conn(0x7f5b140cfc60 msgr2=0x7f5b140d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1259c700 1 --2- 172.21.15.84:0/3242527669 >> v2:172.21.15.107:3300/0 conn(0x7f5b140cfc60 0x7f5b140d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1259c700 1 -- 172.21.15.84:0/3242527669 >> v1:172.21.15.107:6789/0 conn(0x7f5b140d3710 legacy=0x7f5b140cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1259c700 1 -- 172.21.15.84:0/3242527669 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140b5150 con 0x7f5b140b83c0 2023-07-06T21:42:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1259c700 1 --2- 172.21.15.84:0/3242527669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140b83c0 0x7f5b140cd200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b140d0030 tx=0x7f5b04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b4fd0a11cde7bd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1159a700 1 -- 172.21.15.84:0/3242527669 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400b870 con 0x7f5b140b83c0 2023-07-06T21:42:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1159a700 1 -- 172.21.15.84:0/3242527669 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0400b9d0 con 0x7f5b140b83c0 2023-07-06T21:42:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.087+0000 7f5b1159a700 1 -- 172.21.15.84:0/3242527669 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400bd60 con 0x7f5b140b83c0 2023-07-06T21:42:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 -- 172.21.15.84:0/3242527669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140b83c0 msgr2=0x7f5b140cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3242527669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140b83c0 0x7f5b140cd200 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5b140d0030 tx=0x7f5b04009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 -- 172.21.15.84:0/3242527669 shutdown_connections 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3242527669 >> v2:172.21.15.107:3300/0 conn(0x7f5b140cfc60 0x7f5b140d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3242527669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140b83c0 0x7f5b140cd200 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 -- 172.21.15.84:0/3242527669 >> 172.21.15.84:0/3242527669 conn(0x7f5b1406b350 msgr2=0x7f5b1406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 -- 172.21.15.84:0/3242527669 shutdown_connections 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.088+0000 7f5b18bff700 1 -- 172.21.15.84:0/3242527669 wait complete. 2023-07-06T21:42:08.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.090+0000 7f5b18bff700 1 Processor -- start 2023-07-06T21:42:08.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.090+0000 7f5b18bff700 1 -- start start 2023-07-06T21:42:08.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b18bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140b83c0 0x7f5b14155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b18bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140cfc60 0x7f5b140d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b18bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 0x7f5b141f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b18bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141f2fa0 con 0x7f5b140d3710 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b18bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141f29e0 con 0x7f5b140cfc60 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b18bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141f2cc0 con 0x7f5b140b83c0 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 0x7f5b141f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 0x7f5b141f1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54206/0 (socket says 172.21.15.84:54206) 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 -- 172.21.15.84:0/3804949754 learned_addr learned my addr 172.21.15.84:0/3804949754 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b1259c700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140b83c0 0x7f5b14155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 -- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140b83c0 msgr2=0x7f5b14155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140b83c0 0x7f5b14155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 -- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140cfc60 msgr2=0x7f5b140d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140cfc60 0x7f5b140d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.091+0000 7f5b12d9d700 1 -- 172.21.15.84:0/3804949754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b04009550 con 0x7f5b140d3710 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.092+0000 7f5b12d9d700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 0x7f5b141f1f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c007d40 tx=0x7f5b0c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.092+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00ede0 con 0x7f5b140d3710 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.092+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c014420 con 0x7f5b140d3710 2023-07-06T21:42:08.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.092+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14154f90 con 0x7f5b140d3710 2023-07-06T21:42:08.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.092+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c0147d0 con 0x7f5b140d3710 2023-07-06T21:42:08.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.092+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b14155cd0 con 0x7f5b140d3710 2023-07-06T21:42:08.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.093+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14154df0 con 0x7f5b140d3710 2023-07-06T21:42:08.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.094+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b0c014930 con 0x7f5b140d3710 2023-07-06T21:42:08.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.095+0000 7f5b02ffd700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5af406ace0 0x7f5af406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.096+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5b0c089800 con 0x7f5b140d3710 2023-07-06T21:42:08.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.096+0000 7f5b1259c700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5af406ace0 0x7f5af406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.096+0000 7f5b1259c700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5af406ace0 0x7f5af406d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5b1415a050 tx=0x7f5b04008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:08.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.100+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5b14154df0 con 0x7f5b140d3710 2023-07-06T21:42:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.305+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5b14154df0 con 0x7f5b140d3710 2023-07-06T21:42:08.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.306+0000 7f5b02ffd700 1 -- 172.21.15.84:0/3804949754 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) v1 ==== 84+0+126 (secure 0 0 0) 0x7f5b14154df0 con 0x7f5b140d3710 2023-07-06T21:42:08.308 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:08.308 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2023-07-06T21:42:08.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:08.309 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2023-07-06T21:42:08.309 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2023-07-06T21:42:08.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:08.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2023-07-06T21:42:08.309 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 2, 2023-07-06T21:42:08.310 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": "rbd" 2023-07-06T21:42:08.310 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:08.310 INFO:teuthology.orchestra.run.smithi084.stdout:] 2023-07-06T21:42:08.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5af406ace0 msgr2=0x7f5af406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5af406ace0 0x7f5af406d190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5b1415a050 tx=0x7f5b04008030 comp rx=0 tx=0).stop 2023-07-06T21:42:08.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 msgr2=0x7f5b141f1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 0x7f5b141f1f90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c007d40 tx=0x7f5b0c00e3c0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 shutdown_connections 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5af406ace0 0x7f5af406d190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b140d3710 0x7f5b141f1f90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140b83c0 0x7f5b14155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 --2- 172.21.15.84:0/3804949754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140cfc60 0x7f5b140d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 >> 172.21.15.84:0/3804949754 conn(0x7f5b1406b350 msgr2=0x7f5b140d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 shutdown_connections 2023-07-06T21:42:08.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.308+0000 7f5b18bff700 1 -- 172.21.15.84:0/3804949754 wait complete. 2023-07-06T21:42:08.328 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-06T21:42:08.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-06T21:42:08.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.524+0000 7f5b863ff700 1 Processor -- start 2023-07-06T21:42:08.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.524+0000 7f5b863ff700 1 -- start start 2023-07-06T21:42:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b863ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b800cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b863ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b800d0980 0x7f5b800d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b863ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800cd7b0 con 0x7f5b800d2610 2023-07-06T21:42:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b863ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b801609c0 con 0x7f5b800d0980 2023-07-06T21:42:08.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b863ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80160ca0 con 0x7f5b800b4590 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b8499c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5b800d0980 0x7f5b800d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b7f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5b800d2610 0x7f5b800b4960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45190/0 (socket says 172.21.15.84:45190) 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b7f7fe700 1 -- 172.21.15.84:0/674934111 learned_addr learned my addr 172.21.15.84:0/674934111 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.525+0000 7f5b7ffff700 1 --2- 172.21.15.84:0/674934111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b800cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b7effd700 1 -- 172.21.15.84:0/674934111 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225624882 0 0) 0x7f5b800cd7b0 con 0x7f5b800d2610 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b7effd700 1 -- 172.21.15.84:0/674934111 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b680035b0 con 0x7f5b800d2610 2023-07-06T21:42:08.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b8499c700 1 -- 172.21.15.84:0/674934111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 msgr2=0x7f5b800cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b8499c700 1 --2- 172.21.15.84:0/674934111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b800cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b8499c700 1 -- 172.21.15.84:0/674934111 >> v1:172.21.15.84:6789/0 conn(0x7f5b800d2610 legacy=0x7f5b800b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b8499c700 1 -- 172.21.15.84:0/674934111 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800cd160 con 0x7f5b800d0980 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b8499c700 1 --2- 172.21.15.84:0/674934111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d0980 0x7f5b800d0dc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009d90 tx=0x7f5b7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3489ff684d9706e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b7effd700 1 -- 172.21.15.84:0/674934111 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800e070 con 0x7f5b800d0980 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b7effd700 1 -- 172.21.15.84:0/674934111 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800bb20 con 0x7f5b800d0980 2023-07-06T21:42:08.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.526+0000 7f5b7effd700 1 -- 172.21.15.84:0/674934111 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78012430 con 0x7f5b800d0980 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 -- 172.21.15.84:0/674934111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d0980 msgr2=0x7f5b800d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 --2- 172.21.15.84:0/674934111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d0980 0x7f5b800d0dc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009d90 tx=0x7f5b7800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 -- 172.21.15.84:0/674934111 shutdown_connections 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 --2- 172.21.15.84:0/674934111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d0980 0x7f5b800d0dc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 --2- 172.21.15.84:0/674934111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b800cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 -- 172.21.15.84:0/674934111 >> 172.21.15.84:0/674934111 conn(0x7f5b800cf590 msgr2=0x7f5b800cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 -- 172.21.15.84:0/674934111 shutdown_connections 2023-07-06T21:42:08.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.527+0000 7f5b863ff700 1 -- 172.21.15.84:0/674934111 wait complete. 2023-07-06T21:42:08.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.529+0000 7f5b863ff700 1 Processor -- start 2023-07-06T21:42:08.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.529+0000 7f5b863ff700 1 -- start start 2023-07-06T21:42:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b863ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b8015a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b863ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b800d0980 0x7f5b800cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b863ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 0x7f5b800cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b863ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8015d930 con 0x7f5b800d2610 2023-07-06T21:42:08.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b863ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b801e64f0 con 0x7f5b800d0980 2023-07-06T21:42:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b863ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8015d650 con 0x7f5b800b4590 2023-07-06T21:42:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 0x7f5b800cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 0x7f5b800cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54238/0 (socket says 172.21.15.84:54238) 2023-07-06T21:42:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 -- 172.21.15.84:0/3488687494 learned_addr learned my addr 172.21.15.84:0/3488687494 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 -- 172.21.15.84:0/3488687494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 msgr2=0x7f5b8015a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b8015a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 -- 172.21.15.84:0/3488687494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b800d0980 msgr2=0x7f5b800cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b800d0980 0x7f5b800cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.530+0000 7f5b8499c700 1 -- 172.21.15.84:0/3488687494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b78009a70 con 0x7f5b800d2610 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.531+0000 7f5b8499c700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 0x7f5b800cc2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5b78014040 tx=0x7f5b78004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:08.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.531+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b780129f0 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.531+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78020070 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.531+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b801e5e30 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.531+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78012db0 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.531+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8015aab0 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.532+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b801e5c90 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.533+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5b78004040 con 0x7f5b800d2610 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.534+0000 7f5b7cff9700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5b6c06ace0 0x7f5b6c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:08.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.534+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5b78022070 con 0x7f5b800d2610 2023-07-06T21:42:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.534+0000 7f5b7ffff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5b6c06ace0 0x7f5b6c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.535+0000 7f5b7ffff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5b6c06ace0 0x7f5b6c06d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5b70006b20 tx=0x7f5b70006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:08.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.538+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5b801e5c90 con 0x7f5b800d2610 2023-07-06T21:42:08.737 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:08.735+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5b801e6630 con 0x7f5b800d2610 2023-07-06T21:42:09.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.083+0000 7f5b7cff9700 1 -- 172.21.15.84:0/3488687494 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5b801e6630 con 0x7f5b800d2610 2023-07-06T21:42:09.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5b6c06ace0 msgr2=0x7f5b6c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5b6c06ace0 0x7f5b6c06d190 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5b70006b20 tx=0x7f5b70006ab0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 msgr2=0x7f5b800cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 0x7f5b800cc2f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5b78014040 tx=0x7f5b78004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 shutdown_connections 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5b6c06ace0 0x7f5b6c06d190 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5b800d2610 0x7f5b800cc2f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b800b4590 0x7f5b8015a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 --2- 172.21.15.84:0/3488687494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b800d0980 0x7f5b800cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 >> 172.21.15.84:0/3488687494 conn(0x7f5b800cf590 msgr2=0x7f5b800b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:09.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.088+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 shutdown_connections 2023-07-06T21:42:09.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.089+0000 7f5b863ff700 1 -- 172.21.15.84:0/3488687494 wait complete. 2023-07-06T21:42:09.093 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2023-07-06T21:42:09.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-06T21:42:09.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.308+0000 7f385d7ff700 1 Processor -- start 2023-07-06T21:42:09.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.309+0000 7f385d7ff700 1 -- start start 2023-07-06T21:42:09.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f385d7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f385815d7b0 0x7f385815e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:09.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f385d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f385806a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:09.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f385d7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38580c8d50 con 0x7f3858069e90 2023-07-06T21:42:09.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f385d7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385806a870 con 0x7f385815d7b0 2023-07-06T21:42:09.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f385d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385815db80 con 0x7f38580ca7b0 2023-07-06T21:42:09.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f3856ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f385815d7b0 0x7f385815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f384ffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3858069e90 0x7f38580703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45194/0 (socket says 172.21.15.84:45194) 2023-07-06T21:42:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f3856ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f385815d7b0 0x7f385815e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54242/0 (socket says 172.21.15.84:54242) 2023-07-06T21:42:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f384ffff700 1 -- 172.21.15.84:0/2600026656 learned_addr learned my addr 172.21.15.84:0/2600026656 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.310+0000 7f38577fe700 1 --2- 172.21.15.84:0/2600026656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f385806a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f3856ffd700 1 -- 172.21.15.84:0/2600026656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 msgr2=0x7f385806a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:09.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f3856ffd700 1 --2- 172.21.15.84:0/2600026656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f385806a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f3856ffd700 1 -- 172.21.15.84:0/2600026656 >> v1:172.21.15.84:6789/0 conn(0x7f3858069e90 legacy=0x7f38580703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f3856ffd700 1 -- 172.21.15.84:0/2600026656 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38580c9310 con 0x7f385815d7b0 2023-07-06T21:42:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f38567fc700 1 -- 172.21.15.84:0/2600026656 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534035475 0 0) 0x7f38580c8d50 con 0x7f3858069e90 2023-07-06T21:42:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f3856ffd700 1 --2- 172.21.15.84:0/2600026656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f385815e760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3848009840 tx=0x7f3848009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38fb62871c52d2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f38567fc700 1 -- 172.21.15.84:0/2600026656 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848011830 con 0x7f385815d7b0 2023-07-06T21:42:09.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f38567fc700 1 -- 172.21.15.84:0/2600026656 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848011990 con 0x7f385815d7b0 2023-07-06T21:42:09.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.311+0000 7f38567fc700 1 -- 172.21.15.84:0/2600026656 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848011d20 con 0x7f385815d7b0 2023-07-06T21:42:09.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 -- 172.21.15.84:0/2600026656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 msgr2=0x7f385815e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:09.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 --2- 172.21.15.84:0/2600026656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f385815e760 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3848009840 tx=0x7f3848009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:09.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 -- 172.21.15.84:0/2600026656 shutdown_connections 2023-07-06T21:42:09.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 --2- 172.21.15.84:0/2600026656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f385806a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 --2- 172.21.15.84:0/2600026656 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f385815e760 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 -- 172.21.15.84:0/2600026656 >> 172.21.15.84:0/2600026656 conn(0x7f38580cbe10 msgr2=0x7f38580ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 -- 172.21.15.84:0/2600026656 shutdown_connections 2023-07-06T21:42:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.312+0000 7f385d7ff700 1 -- 172.21.15.84:0/2600026656 wait complete. 2023-07-06T21:42:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.314+0000 7f385d7ff700 1 Processor -- start 2023-07-06T21:42:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.314+0000 7f385d7ff700 1 -- start start 2023-07-06T21:42:09.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f385d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3858069e90 0x7f3858153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f385d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f38580b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f385d7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f38580c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f385d7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580b2590 con 0x7f385815d7b0 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f385d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38580aede0 con 0x7f3858069e90 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f385d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38580b1fd0 con 0x7f38580ca7b0 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f38580c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:09.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f38580c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54258/0 (socket says 172.21.15.84:54258) 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 -- 172.21.15.84:0/3364036579 learned_addr learned my addr 172.21.15.84:0/3364036579 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f384ffff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f38580b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f3856ffd700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3858069e90 0x7f3858153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 -- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 msgr2=0x7f38580b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f38580b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 -- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3858069e90 msgr2=0x7f3858153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:09.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3858069e90 0x7f3858153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:09.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.315+0000 7f38577fe700 1 -- 172.21.15.84:0/3364036579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480094a0 con 0x7f385815d7b0 2023-07-06T21:42:09.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.316+0000 7f38577fe700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f38580c46b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f385000efc0 tx=0x7f385000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:09.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.316+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385000cd40 con 0x7f385815d7b0 2023-07-06T21:42:09.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.316+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385000f040 con 0x7f385815d7b0 2023-07-06T21:42:09.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.316+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38580b22b0 con 0x7f385815d7b0 2023-07-06T21:42:09.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.316+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500146c0 con 0x7f385815d7b0 2023-07-06T21:42:09.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.316+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38580c2640 con 0x7f385815d7b0 2023-07-06T21:42:09.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.317+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38580c2860 con 0x7f385815d7b0 2023-07-06T21:42:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.318+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3850014820 con 0x7f385815d7b0 2023-07-06T21:42:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.319+0000 7f384f7fe700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f383806ace0 0x7f383806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.319+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f3850089f20 con 0x7f385815d7b0 2023-07-06T21:42:09.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.319+0000 7f3856ffd700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f383806ace0 0x7f383806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.320+0000 7f3856ffd700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f383806ace0 0x7f383806d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38480090b0 tx=0x7f384801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:09.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.324+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3850058c30 con 0x7f385815d7b0 2023-07-06T21:42:09.521 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:09.519+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f385804e1d0 con 0x7f385815d7b0 2023-07-06T21:42:10.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.087+0000 7f384f7fe700 1 -- 172.21.15.84:0/3364036579 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) v1 ==== 170+0+0 (secure 0 0 0) 0x7f38500180d0 con 0x7f385815d7b0 2023-07-06T21:42:10.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.091+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f383806ace0 msgr2=0x7f383806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.091+0000 7f385d7ff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f383806ace0 0x7f383806d190 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38480090b0 tx=0x7f384801c040 comp rx=0 tx=0).stop 2023-07-06T21:42:10.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.091+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 msgr2=0x7f38580c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.091+0000 7f385d7ff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f38580c46b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f385000efc0 tx=0x7f385000c3a0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.093+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 shutdown_connections 2023-07-06T21:42:10.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.093+0000 7f385d7ff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f383806ace0 0x7f383806d190 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.093+0000 7f385d7ff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f385815d7b0 0x7f38580c46b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.093+0000 7f385d7ff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3858069e90 0x7f3858153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.093+0000 7f385d7ff700 1 --2- 172.21.15.84:0/3364036579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38580ca7b0 0x7f38580b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.093+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 >> 172.21.15.84:0/3364036579 conn(0x7f38580cbe10 msgr2=0x7f38580ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.094+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 shutdown_connections 2023-07-06T21:42:10.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.094+0000 7f385d7ff700 1 -- 172.21.15.84:0/3364036579 wait complete. 2023-07-06T21:42:10.109 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2023-07-06T21:42:10.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-06T21:42:10.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.321+0000 7fedf89ff700 1 Processor -- start 2023-07-06T21:42:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.321+0000 7fedf89ff700 1 -- start start 2023-07-06T21:42:10.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.322+0000 7fedf89ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fedf40b81c0 0x7fedf40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf89ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf40d1830 con 0x7fedf40b1a80 2023-07-06T21:42:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf89ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf4160d40 con 0x7fedf40ceda0 2023-07-06T21:42:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf89ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf40cf330 con 0x7fedf40b81c0 2023-07-06T21:42:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedebfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fedf40b1a80 0x7fedf415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45210/0 (socket says 172.21.15.84:45210) 2023-07-06T21:42:10.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedebfff700 1 -- 172.21.15.84:0/1176339931 learned_addr learned my addr 172.21.15.84:0/1176339931 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf259c700 1 --2- 172.21.15.84:0/1176339931 >> v2:172.21.15.107:3300/0 conn(0x7fedf40b81c0 0x7fedf40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246698458 0 0) 0x7fedf40d1830 con 0x7fedf40b1a80 2023-07-06T21:42:10.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fede00035b0 con 0x7fedf40b1a80 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.323+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339601420 0 0) 0x7fede00035b0 con 0x7fedf40b1a80 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 >> v2:172.21.15.107:3300/0 conn(0x7fedf40b81c0 msgr2=0x7fedf40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 --2- 172.21.15.84:0/1176339931 >> v2:172.21.15.107:3300/0 conn(0x7fedf40b81c0 0x7fedf40cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 >> v1:172.21.15.107:6789/0 conn(0x7fedf40ceda0 legacy=0x7fedf4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40d1270 con 0x7fedf40b1a80 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7feddc001670 con 0x7fedf40b1a80 2023-07-06T21:42:10.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 >> v1:172.21.15.84:6789/0 conn(0x7fedf40b1a80 legacy=0x7fedf415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0003a00 0x7fede0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fede0004850 0x7fede0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fede0005200 0x7fede000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40cf330 con 0x7fede0005200 2023-07-06T21:42:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede000c080 con 0x7fede0004850 2023-07-06T21:42:10.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede000c240 con 0x7fede0003a00 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf259c700 1 --2- 172.21.15.84:0/1176339931 >> v2:172.21.15.107:3300/0 conn(0x7fedf40b81c0 0x7fedf40cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fede0005200 0x7fede000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedebfff700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fede0004850 0x7fede0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf259c700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0003a00 0x7fede0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 -- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0003a00 msgr2=0x7fede0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0003a00 0x7fede0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 -- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fede0004850 msgr2=0x7fede0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fede0004850 0x7fede0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 -- 172.21.15.84:0/1176339931 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede00035b0 con 0x7fede0005200 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.324+0000 7fedf2d9d700 1 -- 172.21.15.84:0/1176339931 reap_dead start 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf2d9d700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fede0005200 0x7fede000ba10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c190 tx=0x7fede0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd76a68c3b897dc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec001db0 con 0x7fede0005200 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedec00ccd0 con 0x7fede0005200 2023-07-06T21:42:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf1d9b700 1 -- 172.21.15.84:0/1176339931 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedec003c40 con 0x7fede0005200 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf89ff700 1 -- 172.21.15.84:0/1176339931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fede0005200 msgr2=0x7fede000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf89ff700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fede0005200 0x7fede000ba10 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fedec00c190 tx=0x7fede0005880 comp rx=0 tx=0).stop 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf89ff700 1 -- 172.21.15.84:0/1176339931 shutdown_connections 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf89ff700 1 --2- 172.21.15.84:0/1176339931 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fede0005200 0x7fede000ba10 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.325+0000 7fedf89ff700 1 -- 172.21.15.84:0/1176339931 >> 172.21.15.84:0/1176339931 conn(0x7fedf406b2b0 msgr2=0x7fedf406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.326+0000 7fedf89ff700 1 -- 172.21.15.84:0/1176339931 shutdown_connections 2023-07-06T21:42:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.326+0000 7fedf89ff700 1 -- 172.21.15.84:0/1176339931 wait complete. 2023-07-06T21:42:10.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.328+0000 7fedf89ff700 1 Processor -- start 2023-07-06T21:42:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.328+0000 7fedf89ff700 1 -- start start 2023-07-06T21:42:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.329+0000 7fedf89ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0005200 0x7fedf41ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.329+0000 7fedf89ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 0x7fedf41f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.329+0000 7fedf89ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf41f3430 0x7fedf40c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.329+0000 7fedf89ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf41ef7d0 con 0x7fedf41f2760 2023-07-06T21:42:10.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.329+0000 7fedf89ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c6a30 con 0x7fedf41f3430 2023-07-06T21:42:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.329+0000 7fedf89ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf40c6d10 con 0x7fede0005200 2023-07-06T21:42:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 0x7fedf41f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 0x7fedf41f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54268/0 (socket says 172.21.15.84:54268) 2023-07-06T21:42:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 -- 172.21.15.84:0/2234470110 learned_addr learned my addr 172.21.15.84:0/2234470110 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedf259c700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0005200 0x7fedf41ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedf2d9d700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf41f3430 0x7fedf40c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 -- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0005200 msgr2=0x7fedf41ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0005200 0x7fedf41ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 -- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf41f3430 msgr2=0x7fedf40c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf41f3430 0x7fedf40c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 -- 172.21.15.84:0/2234470110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf40c6ff0 con 0x7fedf41f2760 2023-07-06T21:42:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedebfff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 0x7fedf41f2ef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7feddc007290 tx=0x7feddc0075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:10.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.330+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc011580 con 0x7fedf41f2760 2023-07-06T21:42:10.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.331+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc0116e0 con 0x7fedf41f2760 2023-07-06T21:42:10.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.331+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf40c72d0 con 0x7fedf41f2760 2023-07-06T21:42:10.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.331+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc011a90 con 0x7fedf41f2760 2023-07-06T21:42:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.331+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf40c6720 con 0x7fedf41f2760 2023-07-06T21:42:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.332+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf40c77f0 con 0x7fedf41f2760 2023-07-06T21:42:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.333+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feddc011bf0 con 0x7fedf41f2760 2023-07-06T21:42:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.334+0000 7fedeaffd700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fedd406ace0 0x7fedd406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.334+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5962+0+0 (secure 0 0 0) 0x7feddc089410 con 0x7fedf41f2760 2023-07-06T21:42:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.334+0000 7fedf259c700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fedd406ace0 0x7fedd406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.334+0000 7fedf259c700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fedd406ace0 0x7fedd406d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fede4009470 tx=0x7fede4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.338+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fedf40c77f0 con 0x7fedf41f2760 2023-07-06T21:42:10.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.576+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fedf40c77f0 con 0x7fedf41f2760 2023-07-06T21:42:10.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.582+0000 7fedeaffd700 1 -- 172.21.15.84:0/2234470110 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fedf40c77f0 con 0x7fedf41f2760 2023-07-06T21:42:10.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.589+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fedd406ace0 msgr2=0x7fedd406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.589+0000 7fedf89ff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fedd406ace0 0x7fedd406d190 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fede4009470 tx=0x7fede4008040 comp rx=0 tx=0).stop 2023-07-06T21:42:10.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.589+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 msgr2=0x7fedf41f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.589+0000 7fedf89ff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 0x7fedf41f2ef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7feddc007290 tx=0x7feddc0075a0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.591+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 shutdown_connections 2023-07-06T21:42:10.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.591+0000 7fedf89ff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fedd406ace0 0x7fedd406d190 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.591+0000 7fedf89ff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedf41f3430 0x7fedf40c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.591+0000 7fedf89ff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fede0005200 0x7fedf41ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.591+0000 7fedf89ff700 1 --2- 172.21.15.84:0/2234470110 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedf41f2760 0x7fedf41f2ef0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.591+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 >> 172.21.15.84:0/2234470110 conn(0x7fedf406b2b0 msgr2=0x7fedf40734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:10.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.596+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 shutdown_connections 2023-07-06T21:42:10.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.601+0000 7fedf89ff700 1 -- 172.21.15.84:0/2234470110 wait complete. 2023-07-06T21:42:10.609 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 8 and data pool 9 2023-07-06T21:42:10.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:10.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.819+0000 7f14041ff700 1 Processor -- start 2023-07-06T21:42:10.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.820+0000 7f14041ff700 1 -- start start 2023-07-06T21:42:10.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f14041ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f13fc0d3710 0x7f13fc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f14041ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc0cd910 con 0x7f13fc0cfa90 2023-07-06T21:42:10.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f14041ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13fc0d0a80 con 0x7f13fc0b83c0 2023-07-06T21:42:10.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f14041ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc0cff10 con 0x7f13fc0d3710 2023-07-06T21:42:10.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f140279c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f13fc0cfa90 0x7f13fc0d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45214/0 (socket says 172.21.15.84:45214) 2023-07-06T21:42:10.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f140279c700 1 -- 172.21.15.84:0/1028373035 learned_addr learned my addr 172.21.15.84:0/1028373035 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:10.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f140179a700 1 --2- 172.21.15.84:0/1028373035 >> v2:172.21.15.107:3300/0 conn(0x7f13fc0d3710 0x7f13fc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383423674 0 0) 0x7f13f4001490 con 0x7f13fc0cfa90 2023-07-06T21:42:10.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.821+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13e80035b0 con 0x7f13fc0cfa90 2023-07-06T21:42:10.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790023656 0 0) 0x7f13fc0d0a80 con 0x7f13fc0b83c0 2023-07-06T21:42:10.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f4001490 con 0x7f13fc0b83c0 2023-07-06T21:42:10.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252712513 0 0) 0x7f13e80035b0 con 0x7f13fc0cfa90 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 >> v2:172.21.15.107:3300/0 conn(0x7f13fc0d3710 msgr2=0x7f13fc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 --2- 172.21.15.84:0/1028373035 >> v2:172.21.15.107:3300/0 conn(0x7f13fc0d3710 0x7f13fc0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 >> v1:172.21.15.107:6789/0 conn(0x7f13fc0b83c0 legacy=0x7f13fc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0b8790 con 0x7f13fc0cfa90 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f13f4001b00 con 0x7f13fc0cfa90 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 >> v1:172.21.15.84:6789/0 conn(0x7f13fc0cfa90 legacy=0x7f13fc0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8003be0 0x7f13e8003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8004850 0x7f13e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13e8005250 0x7f13e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc0cff10 con 0x7f13e8003be0 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e800c160 con 0x7f13e8004850 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e800c320 con 0x7f13e8005250 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1401f9b700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8003be0 0x7f13e8003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f140179a700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8004850 0x7f13e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f140279c700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13e8005250 0x7f13e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1401f9b700 1 -- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13e8005250 msgr2=0x7f13e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1401f9b700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13e8005250 0x7f13e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1401f9b700 1 -- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8004850 msgr2=0x7f13e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.822+0000 7f1401f9b700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8004850 0x7f13e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f1401f9b700 1 -- 172.21.15.84:0/1028373035 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80035b0 con 0x7f13e8003be0 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f140279c700 1 -- 172.21.15.84:0/1028373035 reap_dead start 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f140179a700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8004850 0x7f13e8004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:10.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f1401f9b700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8003be0 0x7f13e8003fd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007fd0 tx=0x7f13e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e03deace812ae83 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8001430 con 0x7f13e8003be0 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80052f0 con 0x7f13e8003be0 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f1400f99700 1 -- 172.21.15.84:0/1028373035 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8001d40 con 0x7f13e8003be0 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f14041ff700 1 -- 172.21.15.84:0/1028373035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8003be0 msgr2=0x7f13e8003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.823+0000 7f14041ff700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8003be0 0x7f13e8003fd0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007fd0 tx=0x7f13e8005880 comp rx=0 tx=0).stop 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.824+0000 7f14041ff700 1 -- 172.21.15.84:0/1028373035 shutdown_connections 2023-07-06T21:42:10.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.824+0000 7f14041ff700 1 --2- 172.21.15.84:0/1028373035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13e8003be0 0x7f13e8003fd0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.824+0000 7f14041ff700 1 -- 172.21.15.84:0/1028373035 >> 172.21.15.84:0/1028373035 conn(0x7f13fc06b350 msgr2=0x7f13fc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:10.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.824+0000 7f14041ff700 1 -- 172.21.15.84:0/1028373035 shutdown_connections 2023-07-06T21:42:10.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.824+0000 7f14041ff700 1 -- 172.21.15.84:0/1028373035 wait complete. 2023-07-06T21:42:10.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.825+0000 7f14041ff700 1 Processor -- start 2023-07-06T21:42:10.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.826+0000 7f14041ff700 1 -- start start 2023-07-06T21:42:10.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.826+0000 7f14041ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8003be0 0x7f13fc1f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.826+0000 7f14041ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13fc0b6af0 0x7f13fc0b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.826+0000 7f14041ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 0x7f13fc153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f14041ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc15f6c0 con 0x7f13fc153700 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f14041ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc15c930 con 0x7f13e8003be0 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f14041ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc15d4b0 con 0x7f13fc0b6af0 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 0x7f13fc153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 0x7f13fc153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54280/0 (socket says 172.21.15.84:54280) 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 -- 172.21.15.84:0/2439972303 learned_addr learned my addr 172.21.15.84:0/2439972303 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:10.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140179a700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13fc0b6af0 0x7f13fc0b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f1401f9b700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8003be0 0x7f13fc1f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 -- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13fc0b6af0 msgr2=0x7f13fc0b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13fc0b6af0 0x7f13fc0b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 -- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8003be0 msgr2=0x7f13fc1f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8003be0 0x7f13fc1f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f140279c700 1 -- 172.21.15.84:0/2439972303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc0b74a0 con 0x7f13fc153700 2023-07-06T21:42:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.827+0000 7f1401f9b700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8003be0 0x7f13fc1f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.828+0000 7f140279c700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 0x7f13fc153ba0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f13f4001bb0 tx=0x7f13f4007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.828+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4017070 con 0x7f13fc153700 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.828+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f400ed50 con 0x7f13fc153700 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.828+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc15cb80 con 0x7f13fc153700 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.828+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4013420 con 0x7f13fc153700 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.828+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc15d140 con 0x7f13fc153700 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.829+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc15d5f0 con 0x7f13fc153700 2023-07-06T21:42:10.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.830+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f13f4013580 con 0x7f13fc153700 2023-07-06T21:42:10.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.831+0000 7f13f27fc700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f13e006ace0 0x7f13e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:10.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.831+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f13f4088f60 con 0x7f13fc153700 2023-07-06T21:42:10.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.831+0000 7f1401f9b700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f13e006ace0 0x7f13e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:10.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.831+0000 7f1401f9b700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f13e006ace0 0x7f13e006d190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005040 tx=0x7f13f800a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:10.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:10.835+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f13f40540a0 con 0x7f13fc153700 2023-07-06T21:42:11.034 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.032+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13fc0cd910 con 0x7f13fc153700 2023-07-06T21:42:11.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.033+0000 7f13f27fc700 1 -- 172.21.15.84:0/2439972303 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f13fc0cd910 con 0x7f13fc153700 2023-07-06T21:42:11.036 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:11.036 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":52,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:10.577943+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:11.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.035+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f13e006ace0 msgr2=0x7f13e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.035+0000 7f14041ff700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f13e006ace0 0x7f13e006d190 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f13f8005040 tx=0x7f13f800a000 comp rx=0 tx=0).stop 2023-07-06T21:42:11.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.035+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 msgr2=0x7f13fc153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.035+0000 7f14041ff700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 0x7f13fc153ba0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f13f4001bb0 tx=0x7f13f4007c90 comp rx=0 tx=0).stop 2023-07-06T21:42:11.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 shutdown_connections 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f13e006ace0 0x7f13e006d190 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13fc153700 0x7f13fc153ba0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e8003be0 0x7f13fc1f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 --2- 172.21.15.84:0/2439972303 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13fc0b6af0 0x7f13fc0b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 >> 172.21.15.84:0/2439972303 conn(0x7f13fc06b350 msgr2=0x7f13fc15bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 shutdown_connections 2023-07-06T21:42:11.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.036+0000 7f14041ff700 1 -- 172.21.15.84:0/2439972303 wait complete. 2023-07-06T21:42:11.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:11.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.249+0000 7f54d2bff700 1 Processor -- start 2023-07-06T21:42:11.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.249+0000 7f54d2bff700 1 -- start start 2023-07-06T21:42:11.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54d2bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 0x7f54cc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54d2bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc0cd910 con 0x7f54cc0b83c0 2023-07-06T21:42:11.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54d2bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc0cfee0 con 0x7f54cc0cfa90 2023-07-06T21:42:11.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54d2bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0d0a80 con 0x7f54cc0d3710 2023-07-06T21:42:11.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54d099b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f54cc0b83c0 0x7f54cc0cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45228/0 (socket says 172.21.15.84:45228) 2023-07-06T21:42:11.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54d099b700 1 -- 172.21.15.84:0/3162829443 learned_addr learned my addr 172.21.15.84:0/3162829443 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:11.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54cbfff700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 0x7f54cc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773178269 0 0) 0x7f54c0001110 con 0x7f54cc0b83c0 2023-07-06T21:42:11.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.251+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54bc0035b0 con 0x7f54cc0b83c0 2023-07-06T21:42:11.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310297272 0 0) 0x7f54bc0035b0 con 0x7f54cc0b83c0 2023-07-06T21:42:11.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 msgr2=0x7f54cc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:11.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 0x7f54cc0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 >> v1:172.21.15.107:6789/0 conn(0x7f54cc0cfa90 legacy=0x7f54cc0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0b8790 con 0x7f54cc0b83c0 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f54c0001800 con 0x7f54cc0b83c0 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 >> v1:172.21.15.84:6789/0 conn(0x7f54cc0b83c0 legacy=0x7f54cc0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54bc003a30 0x7f54bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 0x7f54bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54bc004d90 0x7f54bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0d0a80 con 0x7f54bc004d90 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc00bac0 con 0x7f54bc003a30 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc00bc50 con 0x7f54cc0d3710 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d119c700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54bc004d90 0x7f54bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54cbfff700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 0x7f54bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d099b700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54bc003a30 0x7f54bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d119c700 1 -- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 msgr2=0x7f54bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d119c700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54cc0d3710 0x7f54bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d119c700 1 -- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54bc003a30 msgr2=0x7f54bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d119c700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54bc003a30 0x7f54bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.252+0000 7f54d119c700 1 -- 172.21.15.84:0/3162829443 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0035b0 con 0x7f54bc004d90 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54d119c700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54bc004d90 0x7f54bc005200 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c2a0 tx=0x7f54bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a7ba12e4cc52b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4002080 con 0x7f54bc004d90 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c400ce60 con 0x7f54bc004d90 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54cb7fe700 1 -- 172.21.15.84:0/3162829443 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4012660 con 0x7f54bc004d90 2023-07-06T21:42:11.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54d2bff700 1 -- 172.21.15.84:0/3162829443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54bc004d90 msgr2=0x7f54bc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54d2bff700 1 --2- 172.21.15.84:0/3162829443 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54bc004d90 0x7f54bc005200 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f54c400c2a0 tx=0x7f54bc005880 comp rx=0 tx=0).stop 2023-07-06T21:42:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.253+0000 7f54d119c700 1 -- 172.21.15.84:0/3162829443 reap_dead start 2023-07-06T21:42:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.254+0000 7f54d2bff700 1 -- 172.21.15.84:0/3162829443 shutdown_connections 2023-07-06T21:42:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.254+0000 7f54d2bff700 1 -- 172.21.15.84:0/3162829443 >> 172.21.15.84:0/3162829443 conn(0x7f54cc06b350 msgr2=0x7f54cc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.254+0000 7f54d2bff700 1 -- 172.21.15.84:0/3162829443 shutdown_connections 2023-07-06T21:42:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.254+0000 7f54d2bff700 1 -- 172.21.15.84:0/3162829443 wait complete. 2023-07-06T21:42:11.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.255+0000 7f54d2bff700 1 Processor -- start 2023-07-06T21:42:11.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.256+0000 7f54d2bff700 1 -- start start 2023-07-06T21:42:11.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d2bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54bc004d90 0x7f54cc1f1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d2bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54cc154060 0x7f54cc0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d2bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 0x7f54cc0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d2bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc1f1b60 con 0x7f54cc1544d0 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d2bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc153360 con 0x7f54cc154060 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d2bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc153e50 con 0x7f54bc004d90 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 0x7f54cc0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 0x7f54cc0b3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54304/0 (socket says 172.21.15.84:54304) 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 -- 172.21.15.84:0/2654863864 learned_addr learned my addr 172.21.15.84:0/2654863864 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d099b700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54bc004d90 0x7f54cc1f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54cbfff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54cc154060 0x7f54cc0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 -- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54bc004d90 msgr2=0x7f54cc1f1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54bc004d90 0x7f54cc1f1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 -- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54cc154060 msgr2=0x7f54cc0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54cc154060 0x7f54cc0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.257+0000 7f54d119c700 1 -- 172.21.15.84:0/2654863864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc15ca60 con 0x7f54cc1544d0 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.258+0000 7f54d119c700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 0x7f54cc0b3980 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f54c4013000 tx=0x7f54c4001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.258+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4003e10 con 0x7f54cc1544d0 2023-07-06T21:42:11.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.258+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c40050c0 con 0x7f54cc1544d0 2023-07-06T21:42:11.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.258+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc1535b0 con 0x7f54cc1544d0 2023-07-06T21:42:11.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.258+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4005470 con 0x7f54cc1544d0 2023-07-06T21:42:11.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.258+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc153b70 con 0x7f54cc1544d0 2023-07-06T21:42:11.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.259+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc15cbf0 con 0x7f54cc1544d0 2023-07-06T21:42:11.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.260+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f54c4023070 con 0x7f54cc1544d0 2023-07-06T21:42:11.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.262+0000 7f54c97fa700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54b006ace0 0x7f54b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.262+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f54c408ab50 con 0x7f54cc1544d0 2023-07-06T21:42:11.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.262+0000 7f54d099b700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54b006ace0 0x7f54b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.266 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.263+0000 7f54d099b700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54b006ace0 0x7f54b006d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000c00 tx=0x7f54c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:11.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.266+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f54c4055c90 con 0x7f54cc1544d0 2023-07-06T21:42:11.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.463+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54cc0cfee0 con 0x7f54cc1544d0 2023-07-06T21:42:11.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.464+0000 7f54c97fa700 1 -- 172.21.15.84:0/2654863864 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18360 (secure 0 0 0) 0x7f54cc0cfee0 con 0x7f54cc1544d0 2023-07-06T21:42:11.467 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:11.467 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":52,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:10.577943+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:11.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.466+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54b006ace0 msgr2=0x7f54b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.466+0000 7f54d2bff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54b006ace0 0x7f54b006d190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f54c0000c00 tx=0x7f54c0009040 comp rx=0 tx=0).stop 2023-07-06T21:42:11.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.466+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 msgr2=0x7f54cc0b3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.466+0000 7f54d2bff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 0x7f54cc0b3980 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f54c4013000 tx=0x7f54c4001800 comp rx=0 tx=0).stop 2023-07-06T21:42:11.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.466+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 shutdown_connections 2023-07-06T21:42:11.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f54b006ace0 0x7f54b006d190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54cc1544d0 0x7f54cc0b3980 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54bc004d90 0x7f54cc1f1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 --2- 172.21.15.84:0/2654863864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54cc154060 0x7f54cc0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 >> 172.21.15.84:0/2654863864 conn(0x7f54cc06b350 msgr2=0x7f54cc15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:11.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 shutdown_connections 2023-07-06T21:42:11.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.467+0000 7f54d2bff700 1 -- 172.21.15.84:0/2654863864 wait complete. 2023-07-06T21:42:11.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-06T21:42:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.686+0000 7f89975ff700 1 Processor -- start 2023-07-06T21:42:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.686+0000 7f89975ff700 1 -- start start 2023-07-06T21:42:11.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f89975ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f899006dde0 0x7f899006a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f89975ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89900cf280 0x7f89900b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f89975ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89900c94b0 con 0x7f8990069e90 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f89975ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899006a7a0 con 0x7f899006dde0 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f89975ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89900ca9b0 con 0x7f89900cf280 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f899539b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8990069e90 0x7f899006ce10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:45242/0 (socket says 172.21.15.84:45242) 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f899539b700 1 -- 172.21.15.84:0/3913838659 learned_addr learned my addr 172.21.15.84:0/3913838659 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.687+0000 7f8994b9a700 1 --2- 172.21.15.84:0/3913838659 >> v2:172.21.15.84:3300/0 conn(0x7f899006dde0 0x7f899006a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8995b9c700 1 --2- 172.21.15.84:0/3913838659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89900cf280 0x7f89900b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8987fff700 1 -- 172.21.15.84:0/3913838659 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864014095 0 0) 0x7f89900c94b0 con 0x7f8990069e90 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8987fff700 1 -- 172.21.15.84:0/3913838659 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f897c0035b0 con 0x7f8990069e90 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8994b9a700 1 -- 172.21.15.84:0/3913838659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89900cf280 msgr2=0x7f89900b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8994b9a700 1 --2- 172.21.15.84:0/3913838659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89900cf280 0x7f89900b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8994b9a700 1 -- 172.21.15.84:0/3913838659 >> v1:172.21.15.84:6789/0 conn(0x7f8990069e90 legacy=0x7f899006ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8994b9a700 1 -- 172.21.15.84:0/3913838659 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900cac90 con 0x7f899006dde0 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8994b9a700 1 --2- 172.21.15.84:0/3913838659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899006dde0 0x7f899006a260 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8980009840 tx=0x7f8980009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21415a360e0e5b05 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:11.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.688+0000 7f8987fff700 1 -- 172.21.15.84:0/3913838659 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980011830 con 0x7f899006dde0 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f8987fff700 1 -- 172.21.15.84:0/3913838659 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8980011990 con 0x7f899006dde0 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f8987fff700 1 -- 172.21.15.84:0/3913838659 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8980011d00 con 0x7f899006dde0 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 -- 172.21.15.84:0/3913838659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899006dde0 msgr2=0x7f899006a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 --2- 172.21.15.84:0/3913838659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899006dde0 0x7f899006a260 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f8980009840 tx=0x7f8980009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 -- 172.21.15.84:0/3913838659 shutdown_connections 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 --2- 172.21.15.84:0/3913838659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f89900cf280 0x7f89900b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 --2- 172.21.15.84:0/3913838659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f899006dde0 0x7f899006a260 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 -- 172.21.15.84:0/3913838659 >> 172.21.15.84:0/3913838659 conn(0x7f89900cbf10 msgr2=0x7f89900ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:11.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 -- 172.21.15.84:0/3913838659 shutdown_connections 2023-07-06T21:42:11.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.689+0000 7f89975ff700 1 -- 172.21.15.84:0/3913838659 wait complete. 2023-07-06T21:42:11.696 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.692+0000 7f89975ff700 1 Processor -- start 2023-07-06T21:42:11.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.692+0000 7f89975ff700 1 -- start start 2023-07-06T21:42:11.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f89975ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990069e90 0x7f899015be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f89975ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f899006dde0 0x7f8990151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f89975ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 0x7f899014fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f89975ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899015c360 con 0x7f89900cf280 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f89975ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89901517c0 con 0x7f8990069e90 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f89975ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899015c590 con 0x7f899006dde0 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f8995b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 0x7f899014fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f8995b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 0x7f899014fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54312/0 (socket says 172.21.15.84:54312) 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f8995b9c700 1 -- 172.21.15.84:0/4146040370 learned_addr learned my addr 172.21.15.84:0/4146040370 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:11.698 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.693+0000 7f8994b9a700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f899006dde0 0x7f8990151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f899539b700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990069e90 0x7f899015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8995b9c700 1 -- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f899006dde0 msgr2=0x7f8990151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8995b9c700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f899006dde0 0x7f8990151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8995b9c700 1 -- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990069e90 msgr2=0x7f899015be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8995b9c700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990069e90 0x7f899015be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8995b9c700 1 -- 172.21.15.84:0/4146040370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800094a0 con 0x7f89900cf280 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8995b9c700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 0x7f899014fc20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f898800efc0 tx=0x7f898800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:11.699 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898800cd40 con 0x7f89900cf280 2023-07-06T21:42:11.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898800f040 con 0x7f89900cf280 2023-07-06T21:42:11.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899015c870 con 0x7f89900cf280 2023-07-06T21:42:11.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89880146c0 con 0x7f89900cf280 2023-07-06T21:42:11.700 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.694+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899015ce30 con 0x7f89900cf280 2023-07-06T21:42:11.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.695+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899004e1d0 con 0x7f89900cf280 2023-07-06T21:42:11.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.696+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8988014820 con 0x7f89900cf280 2023-07-06T21:42:11.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.697+0000 7f8985ffb700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f897406ace0 0x7f897406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:11.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.697+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8988089df0 con 0x7f89900cf280 2023-07-06T21:42:11.701 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.697+0000 7f899539b700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f897406ace0 0x7f897406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:11.702 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.698+0000 7f899539b700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f897406ace0 0x7f897406d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f898c005910 tx=0x7f898c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:11.703 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.701+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8988058920 con 0x7f89900cf280 2023-07-06T21:42:11.941 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:11.939+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f899006dba0 con 0x7f89900cf280 2023-07-06T21:42:12.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.589+0000 7f8985ffb700 1 -- 172.21.15.84:0/4146040370 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f898800cea0 con 0x7f89900cf280 2023-07-06T21:42:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f897406ace0 msgr2=0x7f897406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f897406ace0 0x7f897406d190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f898c005910 tx=0x7f898c009040 comp rx=0 tx=0).stop 2023-07-06T21:42:12.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 msgr2=0x7f899014fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 0x7f899014fc20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f898800efc0 tx=0x7f898800c3a0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 shutdown_connections 2023-07-06T21:42:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f897406ace0 0x7f897406d190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89900cf280 0x7f899014fc20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8990069e90 0x7f899015be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 --2- 172.21.15.84:0/4146040370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f899006dde0 0x7f8990151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 >> 172.21.15.84:0/4146040370 conn(0x7f89900cbf10 msgr2=0x7f89900ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 shutdown_connections 2023-07-06T21:42:12.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.592+0000 7f89975ff700 1 -- 172.21.15.84:0/4146040370 wait complete. 2023-07-06T21:42:12.613 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-07-06T21:42:12.614 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-07-06T21:42:12.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.809+0000 7fbdd9bff700 1 Processor -- start 2023-07-06T21:42:12.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.810+0000 7fbdd9bff700 1 -- start start 2023-07-06T21:42:12.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd9bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbdd40b81c0 0x7fbdd40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:12.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 0x7fbdd40cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:12.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd9bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd40d1830 con 0x7fbdd40b1a80 2023-07-06T21:42:12.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd9bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd40cf330 con 0x7fbdd40b81c0 2023-07-06T21:42:12.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd40d1270 con 0x7fbdd40ceda0 2023-07-06T21:42:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 0x7fbdd40cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd37fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbdd40b81c0 0x7fbdd40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd2ffd700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fbdd40b1a80 0x7fbdd415ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:48706/0 (socket says 172.21.15.84:48706) 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 0x7fbdd40cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:50462/0 (socket says 172.21.15.84:50462) 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.811+0000 7fbdd2ffd700 1 -- 172.21.15.84:0/2738583936 learned_addr learned my addr 172.21.15.84:0/2738583936 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd37fe700 1 -- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 msgr2=0x7fbdd40cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd37fe700 1 --2- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 0x7fbdd40cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd37fe700 1 -- 172.21.15.84:0/2738583936 >> v1:172.21.15.107:6789/0 conn(0x7fbdd40b1a80 legacy=0x7fbdd415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd37fe700 1 -- 172.21.15.84:0/2738583936 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd40d1550 con 0x7fbdd40b81c0 2023-07-06T21:42:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd27fc700 1 -- 172.21.15.84:0/2738583936 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932143320 0 0) 0x7fbdd40d1830 con 0x7fbdd40b1a80 2023-07-06T21:42:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd3fff700 1 --2- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 0x7fbdd40cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd37fe700 1 --2- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40cf600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009840 tx=0x7fbdc4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=985e34263fed2e77 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd27fc700 1 -- 172.21.15.84:0/2738583936 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4011830 con 0x7fbdd40b81c0 2023-07-06T21:42:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd27fc700 1 -- 172.21.15.84:0/2738583936 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc4011990 con 0x7fbdd40b81c0 2023-07-06T21:42:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.812+0000 7fbdd27fc700 1 -- 172.21.15.84:0/2738583936 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4011d20 con 0x7fbdd40b81c0 2023-07-06T21:42:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 -- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 msgr2=0x7fbdd40cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40cf600 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009840 tx=0x7fbdc4009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 -- 172.21.15.84:0/2738583936 shutdown_connections 2023-07-06T21:42:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40ceda0 0x7fbdd40cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/2738583936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40cf600 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 -- 172.21.15.84:0/2738583936 >> 172.21.15.84:0/2738583936 conn(0x7fbdd406b2b0 msgr2=0x7fbdd406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 -- 172.21.15.84:0/2738583936 shutdown_connections 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.813+0000 7fbdd9bff700 1 -- 172.21.15.84:0/2738583936 wait complete. 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.815+0000 7fbdd9bff700 1 Processor -- start 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.815+0000 7fbdd9bff700 1 -- start start 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40b1a80 0x7fbdd40c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:12.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd9bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd9bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 0x7fbdd4149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd9bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd414bb60 con 0x7fbdd40ceda0 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd9bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd414af50 con 0x7fbdd40b81c0 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd414b230 con 0x7fbdd40b1a80 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 0x7fbdd4149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 0x7fbdd4149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54318/0 (socket says 172.21.15.84:54318) 2023-07-06T21:42:12.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd3fff700 1 -- 172.21.15.84:0/3786806173 learned_addr learned my addr 172.21.15.84:0/3786806173 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:12.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd2ffd700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:12.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.816+0000 7fbdd37fe700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40b1a80 0x7fbdd40c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:12.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd3fff700 1 -- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40b1a80 msgr2=0x7fbdd40c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:12.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd3fff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40b1a80 0x7fbdd40c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd3fff700 1 -- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 msgr2=0x7fbdd40c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:12.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd3fff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:12.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd3fff700 1 -- 172.21.15.84:0/3786806173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc009a70 con 0x7fbdd40ceda0 2023-07-06T21:42:12.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd2ffd700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:12.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd3fff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 0x7fbdd4149c40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00efc0 tx=0x7fbdcc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:12.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00ca90 con 0x7fbdd40ceda0 2023-07-06T21:42:12.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc00f040 con 0x7fbdd40ceda0 2023-07-06T21:42:12.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc00ce30 con 0x7fbdd40ceda0 2023-07-06T21:42:12.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.817+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40094a0 con 0x7fbdd40ceda0 2023-07-06T21:42:12.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.818+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd414b7f0 con 0x7fbdd40ceda0 2023-07-06T21:42:12.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.818+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd414bd10 con 0x7fbdd40ceda0 2023-07-06T21:42:12.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.820+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbdcc014480 con 0x7fbdd40ceda0 2023-07-06T21:42:12.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.820+0000 7fbdbbfff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbdc006ace0 0x7fbdc006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:12.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.820+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbdcc089ac0 con 0x7fbdd40ceda0 2023-07-06T21:42:12.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.821+0000 7fbdd37fe700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbdc006ace0 0x7fbdc006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:12.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.821+0000 7fbdd37fe700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbdc006ace0 0x7fbdc006d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40090b0 tx=0x7fbdc4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:12.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:12.825+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbdd414bd10 con 0x7fbdd40ceda0 2023-07-06T21:42:13.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.066+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbdd404e1d0 con 0x7fbdd40ceda0 2023-07-06T21:42:13.594 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.592+0000 7fbdbbfff700 1 -- 172.21.15.84:0/3786806173 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbdcc058600 con 0x7fbdd40ceda0 2023-07-06T21:42:13.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbdc006ace0 msgr2=0x7fbdc006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbdc006ace0 0x7fbdc006d190 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40090b0 tx=0x7fbdc4008030 comp rx=0 tx=0).stop 2023-07-06T21:42:13.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 msgr2=0x7fbdd4149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 0x7fbdd4149c40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc00efc0 tx=0x7fbdcc00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 shutdown_connections 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbdc006ace0 0x7fbdc006d190 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbdd40ceda0 0x7fbdd4149c40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdd40b1a80 0x7fbdd40c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 --2- 172.21.15.84:0/3786806173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdd40b81c0 0x7fbdd40c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 >> 172.21.15.84:0/3786806173 conn(0x7fbdd406b2b0 msgr2=0x7fbdd40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:13.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 shutdown_connections 2023-07-06T21:42:13.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.595+0000 7fbdd9bff700 1 -- 172.21.15.84:0/3786806173 wait complete. 2023-07-06T21:42:13.616 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-07-06T21:42:13.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:13.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.807+0000 7f6456dff700 1 Processor -- start 2023-07-06T21:42:13.810 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.808+0000 7f6456dff700 1 -- start start 2023-07-06T21:42:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f6456dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 0x7f6450167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f6456dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64500cd910 con 0x7f64500b83c0 2023-07-06T21:42:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f6456dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64500d0a80 con 0x7f64500d3710 2023-07-06T21:42:13.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f6456dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64500b8790 con 0x7f64500cfc60 2023-07-06T21:42:13.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f6454b9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f64500b83c0 0x7f64500cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56168/0 (socket says 172.21.15.84:56168) 2023-07-06T21:42:13.812 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f6454b9b700 1 -- 172.21.15.84:0/1304473998 learned_addr learned my addr 172.21.15.84:0/1304473998 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:13.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f645539c700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 0x7f6450167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.809+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1208114830 0 0) 0x7f6444001110 con 0x7f64500b83c0 2023-07-06T21:42:13.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64400035b0 con 0x7f64500b83c0 2023-07-06T21:42:13.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116204143 0 0) 0x7f64500d0a80 con 0x7f64500d3710 2023-07-06T21:42:13.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6444001110 con 0x7f64500d3710 2023-07-06T21:42:13.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3125047900 0 0) 0x7f64400035b0 con 0x7f64500b83c0 2023-07-06T21:42:13.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 msgr2=0x7f6450167010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:13.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 0x7f6450167010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 >> v1:172.21.15.107:6789/0 conn(0x7f64500d3710 legacy=0x7f64500cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64500b4e70 con 0x7f64500b83c0 2023-07-06T21:42:13.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f6444001800 con 0x7f64500b83c0 2023-07-06T21:42:13.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 >> v1:172.21.15.84:6789/0 conn(0x7f64500b83c0 legacy=0x7f64500cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6440003bc0 0x7f6440003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6440004850 0x7f6440004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 0x7f6440005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64500b8790 con 0x7f6440003bc0 2023-07-06T21:42:13.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644000bb20 con 0x7f6440004850 2023-07-06T21:42:13.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644000bce0 con 0x7f64500cfc60 2023-07-06T21:42:13.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f6454b9b700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6440003bc0 0x7f6440003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f645539c700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 0x7f6440005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.810+0000 7f644ffff700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6440004850 0x7f6440004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f6454b9b700 1 -- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 msgr2=0x7f6440005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f6454b9b700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f64500cfc60 0x7f6440005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f6454b9b700 1 -- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6440004850 msgr2=0x7f6440004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f6454b9b700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6440004850 0x7f6440004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f6454b9b700 1 -- 172.21.15.84:0/1304473998 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64400035b0 con 0x7f6440003bc0 2023-07-06T21:42:13.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f644ffff700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6440004850 0x7f6440004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f6454b9b700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6440003bc0 0x7f6440003fb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f644400c210 tx=0x7f6440005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96292913f7ed815a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644400cd50 con 0x7f6440003bc0 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6444001c80 con 0x7f6440003bc0 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.811+0000 7f644f7fe700 1 -- 172.21.15.84:0/1304473998 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444005ba0 con 0x7f6440003bc0 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f6456dff700 1 -- 172.21.15.84:0/1304473998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6440003bc0 msgr2=0x7f6440003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f6456dff700 1 --2- 172.21.15.84:0/1304473998 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6440003bc0 0x7f6440003fb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f644400c210 tx=0x7f6440005880 comp rx=0 tx=0).stop 2023-07-06T21:42:13.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f645539c700 1 -- 172.21.15.84:0/1304473998 reap_dead start 2023-07-06T21:42:13.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f6456dff700 1 -- 172.21.15.84:0/1304473998 shutdown_connections 2023-07-06T21:42:13.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f6456dff700 1 -- 172.21.15.84:0/1304473998 >> 172.21.15.84:0/1304473998 conn(0x7f645006b350 msgr2=0x7f645006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:13.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f6456dff700 1 -- 172.21.15.84:0/1304473998 shutdown_connections 2023-07-06T21:42:13.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.812+0000 7f6456dff700 1 -- 172.21.15.84:0/1304473998 wait complete. 2023-07-06T21:42:13.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.814+0000 7f6456dff700 1 Processor -- start 2023-07-06T21:42:13.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.814+0000 7f6456dff700 1 -- start start 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6456dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6440003bc0 0x7f645015f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6456dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f645015c930 0x7f6450151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6456dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 0x7f6450152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6456dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6450156970 con 0x7f6450150790 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6456dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6450151cb0 con 0x7f645015c930 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6456dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6450156060 con 0x7f6440003bc0 2023-07-06T21:42:13.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 0x7f6450152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 0x7f6450152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57294/0 (socket says 172.21.15.84:57294) 2023-07-06T21:42:13.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 -- 172.21.15.84:0/4242050335 learned_addr learned my addr 172.21.15.84:0/4242050335 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:13.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f644ffff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f645015c930 0x7f6450151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f6454b9b700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6440003bc0 0x7f645015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 -- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6440003bc0 msgr2=0x7f645015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6440003bc0 0x7f645015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 -- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f645015c930 msgr2=0x7f6450151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f645015c930 0x7f6450151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f645539c700 1 -- 172.21.15.84:0/4242050335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645015f630 con 0x7f6450150790 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.815+0000 7f644ffff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f645015c930 0x7f6450151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.816+0000 7f645539c700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 0x7f6450152460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f645006c4f0 tx=0x7f644800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.816+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800e040 con 0x7f6450150790 2023-07-06T21:42:13.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.816+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644800f040 con 0x7f6450150790 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.816+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800fc00 con 0x7f6450150790 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.816+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64501562b0 con 0x7f6450150790 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.816+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450150c00 con 0x7f6450150790 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.817+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64500c9a60 con 0x7f6450150790 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.818+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f644800fd60 con 0x7f6450150790 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.819+0000 7f644d7fa700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f643406ace0 0x7f643406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:13.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.819+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6448089b40 con 0x7f6450150790 2023-07-06T21:42:13.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.819+0000 7f6454b9b700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f643406ace0 0x7f643406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:13.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.820+0000 7f6454b9b700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f643406ace0 0x7f643406d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c00 tx=0x7f644400a000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:13.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:13.823+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6448054b50 con 0x7f6450150790 2023-07-06T21:42:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.066+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6450156ab0 con 0x7f6450150790 2023-07-06T21:42:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.066+0000 7f644d7fa700 1 -- 172.21.15.84:0/4242050335 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f6450156ab0 con 0x7f6450150790 2023-07-06T21:42:14.069 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:14.069 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:42:10.578278+0000","modified":"2023-07-06T21:42:13.588255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6834/2341549811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2341549811},{"type":"v1","addr":"172.21.15.107:6834","nonce":2341549811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T21:42:14.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f643406ace0 msgr2=0x7f643406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f643406ace0 0x7f643406d190 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6444000c00 tx=0x7f644400a000 comp rx=0 tx=0).stop 2023-07-06T21:42:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 msgr2=0x7f6450152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 0x7f6450152460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f645006c4f0 tx=0x7f644800f490 comp rx=0 tx=0).stop 2023-07-06T21:42:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 shutdown_connections 2023-07-06T21:42:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f643406ace0 0x7f643406d190 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6450150790 0x7f6450152460 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6440003bc0 0x7f645015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 --2- 172.21.15.84:0/4242050335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f645015c930 0x7f6450151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 >> 172.21.15.84:0/4242050335 conn(0x7f645006b350 msgr2=0x7f64500b6740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 shutdown_connections 2023-07-06T21:42:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.069+0000 7f6456dff700 1 -- 172.21.15.84:0/4242050335 wait complete. 2023-07-06T21:42:14.073 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-07-06T21:42:14.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:14.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.280+0000 7fb28edff700 1 Processor -- start 2023-07-06T21:42:14.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.281+0000 7fb28edff700 1 -- start start 2023-07-06T21:42:14.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb28edff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2880d3710 0x7fb2880cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb28edff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb2880cfa90 0x7fb2880d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb28edff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2880cd910 con 0x7fb2880b83c0 2023-07-06T21:42:14.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb28edff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2880cff10 con 0x7fb2880d3710 2023-07-06T21:42:14.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb28edff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2880d0a80 con 0x7fb2880cfa90 2023-07-06T21:42:14.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb287fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2880d3710 0x7fb2880cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb287fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb2880d3710 0x7fb2880cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57308/0 (socket says 172.21.15.84:57308) 2023-07-06T21:42:14.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb287fff700 1 -- 172.21.15.84:0/1959305756 learned_addr learned my addr 172.21.15.84:0/1959305756 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:14.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.282+0000 7fb28d39c700 1 --2- 172.21.15.84:0/1959305756 >> v2:172.21.15.107:3300/0 conn(0x7fb2880cfa90 0x7fb2880d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb287fff700 1 -- 172.21.15.84:0/1959305756 >> v2:172.21.15.107:3300/0 conn(0x7fb2880cfa90 msgr2=0x7fb2880d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb287fff700 1 --2- 172.21.15.84:0/1959305756 >> v2:172.21.15.107:3300/0 conn(0x7fb2880cfa90 0x7fb2880d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb287fff700 1 -- 172.21.15.84:0/1959305756 >> v1:172.21.15.107:6789/0 conn(0x7fb2880b83c0 legacy=0x7fb2880cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb287fff700 1 -- 172.21.15.84:0/1959305756 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880b8790 con 0x7fb2880d3710 2023-07-06T21:42:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb28d39c700 1 --2- 172.21.15.84:0/1959305756 >> v2:172.21.15.107:3300/0 conn(0x7fb2880cfa90 0x7fb2880d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb287fff700 1 --2- 172.21.15.84:0/1959305756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880cf550 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b1b00 tx=0x7fb274009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa15c9da5099a2ab server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb2877fe700 1 -- 172.21.15.84:0/1959305756 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400b870 con 0x7fb2880d3710 2023-07-06T21:42:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb2877fe700 1 -- 172.21.15.84:0/1959305756 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27400b9d0 con 0x7fb2880d3710 2023-07-06T21:42:14.288 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.283+0000 7fb2877fe700 1 -- 172.21.15.84:0/1959305756 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400bd60 con 0x7fb2880d3710 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 -- 172.21.15.84:0/1959305756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 msgr2=0x7fb2880cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 --2- 172.21.15.84:0/1959305756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880cf550 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb2880b1b00 tx=0x7fb274009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 -- 172.21.15.84:0/1959305756 shutdown_connections 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 --2- 172.21.15.84:0/1959305756 >> v2:172.21.15.107:3300/0 conn(0x7fb2880cfa90 0x7fb2880d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 --2- 172.21.15.84:0/1959305756 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880cf550 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 -- 172.21.15.84:0/1959305756 >> 172.21.15.84:0/1959305756 conn(0x7fb28806b350 msgr2=0x7fb28806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:14.289 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 -- 172.21.15.84:0/1959305756 shutdown_connections 2023-07-06T21:42:14.290 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.284+0000 7fb28edff700 1 -- 172.21.15.84:0/1959305756 wait complete. 2023-07-06T21:42:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.285+0000 7fb28edff700 1 Processor -- start 2023-07-06T21:42:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.286+0000 7fb28edff700 1 -- start start 2023-07-06T21:42:14.291 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.286+0000 7fb28edff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2880b83c0 0x7fb288150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28edff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2880cfa90 0x7fb2880b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28edff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28edff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880c95d0 con 0x7fb2880d3710 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28edff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2881534b0 con 0x7fb2880cfa90 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28edff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb288153790 con 0x7fb2880b83c0 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.292 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880b6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57322/0 (socket says 172.21.15.84:57322) 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 -- 172.21.15.84:0/3236969167 learned_addr learned my addr 172.21.15.84:0/3236969167 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb287fff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2880cfa90 0x7fb2880b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28cb9b700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2880b83c0 0x7fb288150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 -- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2880b83c0 msgr2=0x7fb288150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2880b83c0 0x7fb288150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 -- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2880cfa90 msgr2=0x7fb2880b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2880cfa90 0x7fb2880b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.293 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 -- 172.21.15.84:0/3236969167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb274009550 con 0x7fb2880d3710 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.287+0000 7fb28d39c700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880b6f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb280007e70 tx=0x7fb28000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.288+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000ddd0 con 0x7fb2880d3710 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.288+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28000f040 con 0x7fb2880d3710 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.288+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb288153a70 con 0x7fb2880d3710 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.288+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280003c60 con 0x7fb2880d3710 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.288+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288154030 con 0x7fb2880d3710 2023-07-06T21:42:14.294 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.289+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb288154220 con 0x7fb2880d3710 2023-07-06T21:42:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.290+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb280004030 con 0x7fb2880d3710 2023-07-06T21:42:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.291+0000 7fb2857fa700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb26c06ace0 0x7fb26c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.291+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb280089680 con 0x7fb2880d3710 2023-07-06T21:42:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.291+0000 7fb28cb9b700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb26c06ace0 0x7fb26c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.296 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.292+0000 7fb28cb9b700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb26c06ace0 0x7fb26c06d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb27c000f60 tx=0x7fb27c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:14.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.295+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb280054600 con 0x7fb2880d3710 2023-07-06T21:42:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.492+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb28815b090 con 0x7fb2880d3710 2023-07-06T21:42:14.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.493+0000 7fb2857fa700 1 -- 172.21.15.84:0/3236969167 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7fb280057c20 con 0x7fb2880d3710 2023-07-06T21:42:14.496 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:14.497 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:14.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.495+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb26c06ace0 msgr2=0x7fb26c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.495+0000 7fb28edff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb26c06ace0 0x7fb26c06d190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb27c000f60 tx=0x7fb27c009040 comp rx=0 tx=0).stop 2023-07-06T21:42:14.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.495+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 msgr2=0x7fb2880b6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.495+0000 7fb28edff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880b6f00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb280007e70 tx=0x7fb28000d3b0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.495+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 shutdown_connections 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.495+0000 7fb28edff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb26c06ace0 0x7fb26c06d190 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.496+0000 7fb28edff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb2880d3710 0x7fb2880b6f00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.496+0000 7fb28edff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2880b83c0 0x7fb288150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.496+0000 7fb28edff700 1 --2- 172.21.15.84:0/3236969167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2880cfa90 0x7fb2880b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.496+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 >> 172.21.15.84:0/3236969167 conn(0x7fb28806b350 msgr2=0x7fb2880d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.496+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 shutdown_connections 2023-07-06T21:42:14.499 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.496+0000 7fb28edff700 1 -- 172.21.15.84:0/3236969167 wait complete. 2023-07-06T21:42:14.516 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-06T21:42:14.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:42:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.708+0000 7f2f2cfff700 1 Processor -- start 2023-07-06T21:42:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.708+0000 7f2f2cfff700 1 -- start start 2023-07-06T21:42:14.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.709+0000 7f2f2cfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280b2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.709+0000 7f2f2cfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2806b8a0 con 0x7f2f2815c6a0 2023-07-06T21:42:14.711 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.709+0000 7f2f2cfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2806bb80 con 0x7f2f280b3010 2023-07-06T21:42:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.709+0000 7f2f2cfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2806b6b0 con 0x7f2f280b2470 2023-07-06T21:42:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.709+0000 7f2f2659c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f2f2815c6a0 0x7f2f2806c940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56180/0 (socket says 172.21.15.84:56180) 2023-07-06T21:42:14.712 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.709+0000 7f2f2659c700 1 -- 172.21.15.84:0/3708836335 learned_addr learned my addr 172.21.15.84:0/3708836335 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:14.713 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f26d9d700 1 --2- 172.21.15.84:0/3708836335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134057405 0 0) 0x7f2f2806b8a0 con 0x7f2f2815c6a0 2023-07-06T21:42:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f140035b0 con 0x7f2f2815c6a0 2023-07-06T21:42:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123377482 0 0) 0x7f2f2806bb80 con 0x7f2f280b3010 2023-07-06T21:42:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2806b8a0 con 0x7f2f280b3010 2023-07-06T21:42:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1245229349 0 0) 0x7f2f140035b0 con 0x7f2f2815c6a0 2023-07-06T21:42:14.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f26d9d700 1 -- 172.21.15.84:0/3708836335 >> v1:172.21.15.107:6789/0 conn(0x7f2f280b3010 legacy=0x7f2f2806afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f26d9d700 1 -- 172.21.15.84:0/3708836335 >> v1:172.21.15.84:6789/0 conn(0x7f2f2815c6a0 legacy=0x7f2f2806c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f26d9d700 1 -- 172.21.15.84:0/3708836335 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2806be60 con 0x7f2f280b2470 2023-07-06T21:42:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.710+0000 7f2f26d9d700 1 --2- 172.21.15.84:0/3708836335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280b2910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009870 tx=0x7f2f20009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcaeaa1e6b90aae server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f180019f0 con 0x7f2f280b2470 2023-07-06T21:42:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f20004030 con 0x7f2f280b2470 2023-07-06T21:42:14.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f25d9b700 1 -- 172.21.15.84:0/3708836335 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f200043a0 con 0x7f2f280b2470 2023-07-06T21:42:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3708836335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 msgr2=0x7f2f280b2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3708836335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280b2910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009870 tx=0x7f2f20009b80 comp rx=0 tx=0).stop 2023-07-06T21:42:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3708836335 shutdown_connections 2023-07-06T21:42:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3708836335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280b2910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3708836335 >> 172.21.15.84:0/3708836335 conn(0x7f2f2806e640 msgr2=0x7f2f2806ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3708836335 shutdown_connections 2023-07-06T21:42:14.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.711+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3708836335 wait complete. 2023-07-06T21:42:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.713+0000 7f2f2cfff700 1 Processor -- start 2023-07-06T21:42:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.713+0000 7f2f2cfff700 1 -- start start 2023-07-06T21:42:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f2cfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f2cfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 0x7f2f28157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f2cfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f2815c6a0 0x7f2f280b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f2cfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f28158260 con 0x7f2f280b3010 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f2cfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280c5a40 con 0x7f2f2815c6a0 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f2cfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2815e370 con 0x7f2f280b2470 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 0x7f2f28157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f1ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 0x7f2f28157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57332/0 (socket says 172.21.15.84:57332) 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.714+0000 7f2f1ffff700 1 -- 172.21.15.84:0/3360087448 learned_addr learned my addr 172.21.15.84:0/3360087448 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f26d9d700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f2815c6a0 0x7f2f280b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f2659c700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1ffff700 1 -- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 msgr2=0x7f2f280c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1ffff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1ffff700 1 -- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f2815c6a0 msgr2=0x7f2f280b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1ffff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f2815c6a0 0x7f2f280b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1ffff700 1 -- 172.21.15.84:0/3360087448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f200094a0 con 0x7f2f280b3010 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f26d9d700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f2815c6a0 0x7f2f280b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f2659c700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1ffff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 0x7f2f28157d20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c520 tx=0x7f2f1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f100052d0 con 0x7f2f280b3010 2023-07-06T21:42:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f10005430 con 0x7f2f280b3010 2023-07-06T21:42:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f100038c0 con 0x7f2f280b3010 2023-07-06T21:42:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2815e650 con 0x7f2f280b3010 2023-07-06T21:42:14.722 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.715+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f280b65d0 con 0x7f2f280b3010 2023-07-06T21:42:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.716+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f280709c0 con 0x7f2f280b3010 2023-07-06T21:42:14.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.717+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f2f10003a20 con 0x7f2f280b3010 2023-07-06T21:42:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.718+0000 7f2f1effd700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f0806ace0 0x7f2f0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.718+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f10088af0 con 0x7f2f280b3010 2023-07-06T21:42:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.718+0000 7f2f2659c700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f0806ace0 0x7f2f0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.719+0000 7f2f2659c700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f0806ace0 0x7f2f0806d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800d150 tx=0x7f2f18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.722+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f2f10053b00 con 0x7f2f280b3010 2023-07-06T21:42:14.915 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.914+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2f2806c420 con 0x7f2f0806ace0 2023-07-06T21:42:14.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.924+0000 7f2f1effd700 1 -- 172.21.15.84:0/3360087448 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311283 (secure 0 0 0) 0x7f2f1800daf0 con 0x7f2f0806ace0 2023-07-06T21:42:14.930 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:14.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f0806ace0 msgr2=0x7f2f0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f0806ace0 0x7f2f0806d190 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800d150 tx=0x7f2f18009040 comp rx=0 tx=0).stop 2023-07-06T21:42:14.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 msgr2=0x7f2f28157d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 0x7f2f28157d20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000c520 tx=0x7f2f1000c8e0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 shutdown_connections 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f2f0806ace0 0x7f2f0806d190 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f2815c6a0 0x7f2f280b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f280b2470 0x7f2f280c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 --2- 172.21.15.84:0/3360087448 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2f280b3010 0x7f2f28157d20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:14.934 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 >> 172.21.15.84:0/3360087448 conn(0x7f2f2806e640 msgr2=0x7f2f280b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:14.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 shutdown_connections 2023-07-06T21:42:14.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:14.928+0000 7f2f2cfff700 1 -- 172.21.15.84:0/3360087448 wait complete. 2023-07-06T21:42:14.935 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:42:14.949 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2023-07-06T21:42:14.170767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221692,"kb_used_data":6076,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754753028,"statfs":{"total":773094113280,"available":772867100672,"internally_reserved":0,"allocated":6221824,"data_stored":3355331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.087877"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589343+0000","last_change":"2023-07-06T21:42:10.595808+0000","last_active":"2023-07-06T21:42:11.589343+0000","last_peered":"2023-07-06T21:42:11.589343+0000","last_clean":"2023-07-06T21:42:11.589343+0000","last_became_active":"2023-07-06T21:42:10.595536+0000","last_became_peered":"2023-07-06T21:42:10.595536+0000","last_unstale":"2023-07-06T21:42:11.589343+0000","last_undegraded":"2023-07-06T21:42:11.589343+0000","last_fullsized":"2023-07-06T21:42:11.589343+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:17:07.073382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.599197+0000","last_change":"2023-07-06T21:42:10.594349+0000","last_active":"2023-07-06T21:42:10.599197+0000","last_peered":"2023-07-06T21:42:10.599197+0000","last_clean":"2023-07-06T21:42:10.599197+0000","last_became_active":"2023-07-06T21:42:10.594082+0000","last_became_peered":"2023-07-06T21:42:10.594082+0000","last_unstale":"2023-07-06T21:42:10.599197+0000","last_undegraded":"2023-07-06T21:42:10.599197+0000","last_fullsized":"2023-07-06T21:42:10.599197+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:59:59.104906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619441+0000","last_change":"2023-07-06T21:42:10.610344+0000","last_active":"2023-07-06T21:42:10.619441+0000","last_peered":"2023-07-06T21:42:10.619441+0000","last_clean":"2023-07-06T21:42:10.619441+0000","last_became_active":"2023-07-06T21:42:10.608807+0000","last_became_peered":"2023-07-06T21:42:10.608807+0000","last_unstale":"2023-07-06T21:42:10.619441+0000","last_undegraded":"2023-07-06T21:42:10.619441+0000","last_fullsized":"2023-07-06T21:42:10.619441+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:37:03.501526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590759+0000","last_change":"2023-07-06T21:42:10.601610+0000","last_active":"2023-07-06T21:42:11.590759+0000","last_peered":"2023-07-06T21:42:11.590759+0000","last_clean":"2023-07-06T21:42:11.590759+0000","last_became_active":"2023-07-06T21:42:10.601222+0000","last_became_peered":"2023-07-06T21:42:10.601222+0000","last_unstale":"2023-07-06T21:42:11.590759+0000","last_undegraded":"2023-07-06T21:42:11.590759+0000","last_fullsized":"2023-07-06T21:42:11.590759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:13:15.832150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589160+0000","last_change":"2023-07-06T21:42:10.605718+0000","last_active":"2023-07-06T21:42:11.589160+0000","last_peered":"2023-07-06T21:42:11.589160+0000","last_clean":"2023-07-06T21:42:11.589160+0000","last_became_active":"2023-07-06T21:42:10.605414+0000","last_became_peered":"2023-07-06T21:42:10.605414+0000","last_unstale":"2023-07-06T21:42:11.589160+0000","last_undegraded":"2023-07-06T21:42:11.589160+0000","last_fullsized":"2023-07-06T21:42:11.589160+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:14:27.518098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.601551+0000","last_change":"2023-07-06T21:42:10.598487+0000","last_active":"2023-07-06T21:42:10.601551+0000","last_peered":"2023-07-06T21:42:10.601551+0000","last_clean":"2023-07-06T21:42:10.601551+0000","last_became_active":"2023-07-06T21:42:10.598234+0000","last_became_peered":"2023-07-06T21:42:10.598234+0000","last_unstale":"2023-07-06T21:42:10.601551+0000","last_undegraded":"2023-07-06T21:42:10.601551+0000","last_fullsized":"2023-07-06T21:42:10.601551+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:44:30.629246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590827+0000","last_change":"2023-07-06T21:42:10.604054+0000","last_active":"2023-07-06T21:42:11.590827+0000","last_peered":"2023-07-06T21:42:11.590827+0000","last_clean":"2023-07-06T21:42:11.590827+0000","last_became_active":"2023-07-06T21:42:10.603787+0000","last_became_peered":"2023-07-06T21:42:10.603787+0000","last_unstale":"2023-07-06T21:42:11.590827+0000","last_undegraded":"2023-07-06T21:42:11.590827+0000","last_fullsized":"2023-07-06T21:42:11.590827+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:58:16.849785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.608841+0000","last_change":"2023-07-06T21:42:10.606029+0000","last_active":"2023-07-06T21:42:10.608841+0000","last_peered":"2023-07-06T21:42:10.608841+0000","last_clean":"2023-07-06T21:42:10.608841+0000","last_became_active":"2023-07-06T21:42:10.604342+0000","last_became_peered":"2023-07-06T21:42:10.604342+0000","last_unstale":"2023-07-06T21:42:10.608841+0000","last_undegraded":"2023-07-06T21:42:10.608841+0000","last_fullsized":"2023-07-06T21:42:10.608841+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:40:27.493052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591675+0000","last_change":"2023-07-06T21:42:10.605090+0000","last_active":"2023-07-06T21:42:11.591675+0000","last_peered":"2023-07-06T21:42:11.591675+0000","last_clean":"2023-07-06T21:42:11.591675+0000","last_became_active":"2023-07-06T21:42:10.604757+0000","last_became_peered":"2023-07-06T21:42:10.604757+0000","last_unstale":"2023-07-06T21:42:11.591675+0000","last_undegraded":"2023-07-06T21:42:11.591675+0000","last_fullsized":"2023-07-06T21:42:11.591675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:55.967710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.611194+0000","last_change":"2023-07-06T21:42:10.605227+0000","last_active":"2023-07-06T21:42:10.611194+0000","last_peered":"2023-07-06T21:42:10.611194+0000","last_clean":"2023-07-06T21:42:10.611194+0000","last_became_active":"2023-07-06T21:42:10.604932+0000","last_became_peered":"2023-07-06T21:42:10.604932+0000","last_unstale":"2023-07-06T21:42:10.611194+0000","last_undegraded":"2023-07-06T21:42:10.611194+0000","last_fullsized":"2023-07-06T21:42:10.611194+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:32.056656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.603801+0000","last_change":"2023-07-06T21:42:10.595947+0000","last_active":"2023-07-06T21:42:10.603801+0000","last_peered":"2023-07-06T21:42:10.603801+0000","last_clean":"2023-07-06T21:42:10.603801+0000","last_became_active":"2023-07-06T21:42:10.595698+0000","last_became_peered":"2023-07-06T21:42:10.595698+0000","last_unstale":"2023-07-06T21:42:10.603801+0000","last_undegraded":"2023-07-06T21:42:10.603801+0000","last_fullsized":"2023-07-06T21:42:10.603801+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:03:10.800847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590917+0000","last_change":"2023-07-06T21:42:10.606844+0000","last_active":"2023-07-06T21:42:11.590917+0000","last_peered":"2023-07-06T21:42:11.590917+0000","last_clean":"2023-07-06T21:42:11.590917+0000","last_became_active":"2023-07-06T21:42:10.606471+0000","last_became_peered":"2023-07-06T21:42:10.606471+0000","last_unstale":"2023-07-06T21:42:11.590917+0000","last_undegraded":"2023-07-06T21:42:11.590917+0000","last_fullsized":"2023-07-06T21:42:11.590917+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:48.938598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606204+0000","last_change":"2023-07-06T21:42:10.602650+0000","last_active":"2023-07-06T21:42:10.606204+0000","last_peered":"2023-07-06T21:42:10.606204+0000","last_clean":"2023-07-06T21:42:10.606204+0000","last_became_active":"2023-07-06T21:42:10.602364+0000","last_became_peered":"2023-07-06T21:42:10.602364+0000","last_unstale":"2023-07-06T21:42:10.606204+0000","last_undegraded":"2023-07-06T21:42:10.606204+0000","last_fullsized":"2023-07-06T21:42:10.606204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:14:26.542391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591081+0000","last_change":"2023-07-06T21:42:10.598716+0000","last_active":"2023-07-06T21:42:11.591081+0000","last_peered":"2023-07-06T21:42:11.591081+0000","last_clean":"2023-07-06T21:42:11.591081+0000","last_became_active":"2023-07-06T21:42:10.598418+0000","last_became_peered":"2023-07-06T21:42:10.598418+0000","last_unstale":"2023-07-06T21:42:11.591081+0000","last_undegraded":"2023-07-06T21:42:11.591081+0000","last_fullsized":"2023-07-06T21:42:11.591081+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:10.542591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590892+0000","last_change":"2023-07-06T21:42:10.598485+0000","last_active":"2023-07-06T21:42:11.590892+0000","last_peered":"2023-07-06T21:42:11.590892+0000","last_clean":"2023-07-06T21:42:11.590892+0000","last_became_active":"2023-07-06T21:42:10.598173+0000","last_became_peered":"2023-07-06T21:42:10.598173+0000","last_unstale":"2023-07-06T21:42:11.590892+0000","last_undegraded":"2023-07-06T21:42:11.590892+0000","last_fullsized":"2023-07-06T21:42:11.590892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:30.099898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612496+0000","last_change":"2023-07-06T21:42:10.604690+0000","last_active":"2023-07-06T21:42:10.612496+0000","last_peered":"2023-07-06T21:42:10.612496+0000","last_clean":"2023-07-06T21:42:10.612496+0000","last_became_active":"2023-07-06T21:42:10.603767+0000","last_became_peered":"2023-07-06T21:42:10.603767+0000","last_unstale":"2023-07-06T21:42:10.612496+0000","last_undegraded":"2023-07-06T21:42:10.612496+0000","last_fullsized":"2023-07-06T21:42:10.612496+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:41:59.331696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.608784+0000","last_change":"2023-07-06T21:42:10.604339+0000","last_active":"2023-07-06T21:42:10.608784+0000","last_peered":"2023-07-06T21:42:10.608784+0000","last_clean":"2023-07-06T21:42:10.608784+0000","last_became_active":"2023-07-06T21:42:10.603872+0000","last_became_peered":"2023-07-06T21:42:10.603872+0000","last_unstale":"2023-07-06T21:42:10.608784+0000","last_undegraded":"2023-07-06T21:42:10.608784+0000","last_fullsized":"2023-07-06T21:42:10.608784+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:55.967710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619625+0000","last_change":"2023-07-06T21:42:10.613543+0000","last_active":"2023-07-06T21:42:10.619625+0000","last_peered":"2023-07-06T21:42:10.619625+0000","last_clean":"2023-07-06T21:42:10.619625+0000","last_became_active":"2023-07-06T21:42:10.613166+0000","last_became_peered":"2023-07-06T21:42:10.613166+0000","last_unstale":"2023-07-06T21:42:10.619625+0000","last_undegraded":"2023-07-06T21:42:10.619625+0000","last_fullsized":"2023-07-06T21:42:10.619625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:27:00.253582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589066+0000","last_change":"2023-07-06T21:42:10.595764+0000","last_active":"2023-07-06T21:42:11.589066+0000","last_peered":"2023-07-06T21:42:11.589066+0000","last_clean":"2023-07-06T21:42:11.589066+0000","last_became_active":"2023-07-06T21:42:10.595468+0000","last_became_peered":"2023-07-06T21:42:10.595468+0000","last_unstale":"2023-07-06T21:42:11.589066+0000","last_undegraded":"2023-07-06T21:42:11.589066+0000","last_fullsized":"2023-07-06T21:42:11.589066+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:40:43.998094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609274+0000","last_change":"2023-07-06T21:42:10.607771+0000","last_active":"2023-07-06T21:42:10.609274+0000","last_peered":"2023-07-06T21:42:10.609274+0000","last_clean":"2023-07-06T21:42:10.609274+0000","last_became_active":"2023-07-06T21:42:10.607449+0000","last_became_peered":"2023-07-06T21:42:10.607449+0000","last_unstale":"2023-07-06T21:42:10.609274+0000","last_undegraded":"2023-07-06T21:42:10.609274+0000","last_fullsized":"2023-07-06T21:42:10.609274+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:38:06.944141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.620475+0000","last_change":"2023-07-06T21:42:10.616730+0000","last_active":"2023-07-06T21:42:10.620475+0000","last_peered":"2023-07-06T21:42:10.620475+0000","last_clean":"2023-07-06T21:42:10.620475+0000","last_became_active":"2023-07-06T21:42:10.616383+0000","last_became_peered":"2023-07-06T21:42:10.616383+0000","last_unstale":"2023-07-06T21:42:10.620475+0000","last_undegraded":"2023-07-06T21:42:10.620475+0000","last_fullsized":"2023-07-06T21:42:10.620475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:30:44.679449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606375+0000","last_change":"2023-07-06T21:42:10.598413+0000","last_active":"2023-07-06T21:42:10.606375+0000","last_peered":"2023-07-06T21:42:10.606375+0000","last_clean":"2023-07-06T21:42:10.606375+0000","last_became_active":"2023-07-06T21:42:10.597804+0000","last_became_peered":"2023-07-06T21:42:10.597804+0000","last_unstale":"2023-07-06T21:42:10.606375+0000","last_undegraded":"2023-07-06T21:42:10.606375+0000","last_fullsized":"2023-07-06T21:42:10.606375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:25.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.615501+0000","last_change":"2023-07-06T21:42:10.610209+0000","last_active":"2023-07-06T21:42:10.615501+0000","last_peered":"2023-07-06T21:42:10.615501+0000","last_clean":"2023-07-06T21:42:10.615501+0000","last_became_active":"2023-07-06T21:42:10.609884+0000","last_became_peered":"2023-07-06T21:42:10.609884+0000","last_unstale":"2023-07-06T21:42:10.615501+0000","last_undegraded":"2023-07-06T21:42:10.615501+0000","last_fullsized":"2023-07-06T21:42:10.615501+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:56:24.004393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612366+0000","last_change":"2023-07-06T21:42:10.607800+0000","last_active":"2023-07-06T21:42:10.612366+0000","last_peered":"2023-07-06T21:42:10.612366+0000","last_clean":"2023-07-06T21:42:10.612366+0000","last_became_active":"2023-07-06T21:42:10.607299+0000","last_became_peered":"2023-07-06T21:42:10.607299+0000","last_unstale":"2023-07-06T21:42:10.612366+0000","last_undegraded":"2023-07-06T21:42:10.612366+0000","last_fullsized":"2023-07-06T21:42:10.612366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:11.235771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589313+0000","last_change":"2023-07-06T21:42:10.594998+0000","last_active":"2023-07-06T21:42:11.589313+0000","last_peered":"2023-07-06T21:42:11.589313+0000","last_clean":"2023-07-06T21:42:11.589313+0000","last_became_active":"2023-07-06T21:42:10.594743+0000","last_became_peered":"2023-07-06T21:42:10.594743+0000","last_unstale":"2023-07-06T21:42:11.589313+0000","last_undegraded":"2023-07-06T21:42:11.589313+0000","last_fullsized":"2023-07-06T21:42:11.589313+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:24.540760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591427+0000","last_change":"2023-07-06T21:42:10.608043+0000","last_active":"2023-07-06T21:42:11.591427+0000","last_peered":"2023-07-06T21:42:11.591427+0000","last_clean":"2023-07-06T21:42:11.591427+0000","last_became_active":"2023-07-06T21:42:10.607743+0000","last_became_peered":"2023-07-06T21:42:10.607743+0000","last_unstale":"2023-07-06T21:42:11.591427+0000","last_undegraded":"2023-07-06T21:42:11.591427+0000","last_fullsized":"2023-07-06T21:42:11.591427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:56:28.649508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612463+0000","last_change":"2023-07-06T21:42:10.608269+0000","last_active":"2023-07-06T21:42:10.612463+0000","last_peered":"2023-07-06T21:42:10.612463+0000","last_clean":"2023-07-06T21:42:10.612463+0000","last_became_active":"2023-07-06T21:42:10.608013+0000","last_became_peered":"2023-07-06T21:42:10.608013+0000","last_unstale":"2023-07-06T21:42:10.612463+0000","last_undegraded":"2023-07-06T21:42:10.612463+0000","last_fullsized":"2023-07-06T21:42:10.612463+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:03:10.800847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589114+0000","last_change":"2023-07-06T21:42:10.605574+0000","last_active":"2023-07-06T21:42:11.589114+0000","last_peered":"2023-07-06T21:42:11.589114+0000","last_clean":"2023-07-06T21:42:11.589114+0000","last_became_active":"2023-07-06T21:42:10.605287+0000","last_became_peered":"2023-07-06T21:42:10.605287+0000","last_unstale":"2023-07-06T21:42:11.589114+0000","last_undegraded":"2023-07-06T21:42:11.589114+0000","last_fullsized":"2023-07-06T21:42:11.589114+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:01.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612162+0000","last_change":"2023-07-06T21:42:10.610723+0000","last_active":"2023-07-06T21:42:10.612162+0000","last_peered":"2023-07-06T21:42:10.612162+0000","last_clean":"2023-07-06T21:42:10.612162+0000","last_became_active":"2023-07-06T21:42:10.609369+0000","last_became_peered":"2023-07-06T21:42:10.609369+0000","last_unstale":"2023-07-06T21:42:10.612162+0000","last_undegraded":"2023-07-06T21:42:10.612162+0000","last_fullsized":"2023-07-06T21:42:10.612162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:14.024714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619487+0000","last_change":"2023-07-06T21:42:10.610652+0000","last_active":"2023-07-06T21:42:10.619487+0000","last_peered":"2023-07-06T21:42:10.619487+0000","last_clean":"2023-07-06T21:42:10.619487+0000","last_became_active":"2023-07-06T21:42:10.610322+0000","last_became_peered":"2023-07-06T21:42:10.610322+0000","last_unstale":"2023-07-06T21:42:10.619487+0000","last_undegraded":"2023-07-06T21:42:10.619487+0000","last_fullsized":"2023-07-06T21:42:10.619487+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:12:01.721085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.588918+0000","last_change":"2023-07-06T21:42:10.593788+0000","last_active":"2023-07-06T21:42:11.588918+0000","last_peered":"2023-07-06T21:42:11.588918+0000","last_clean":"2023-07-06T21:42:11.588918+0000","last_became_active":"2023-07-06T21:42:10.593514+0000","last_became_peered":"2023-07-06T21:42:10.593514+0000","last_unstale":"2023-07-06T21:42:11.588918+0000","last_undegraded":"2023-07-06T21:42:11.588918+0000","last_fullsized":"2023-07-06T21:42:11.588918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:29.124985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.604539+0000","last_change":"2023-07-06T21:42:10.599454+0000","last_active":"2023-07-06T21:42:10.604539+0000","last_peered":"2023-07-06T21:42:10.604539+0000","last_clean":"2023-07-06T21:42:10.604539+0000","last_became_active":"2023-07-06T21:42:10.599126+0000","last_became_peered":"2023-07-06T21:42:10.599126+0000","last_unstale":"2023-07-06T21:42:10.604539+0000","last_undegraded":"2023-07-06T21:42:10.604539+0000","last_fullsized":"2023-07-06T21:42:10.604539+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:59:27.262227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612512+0000","last_change":"2023-07-06T21:42:10.609066+0000","last_active":"2023-07-06T21:42:10.612512+0000","last_peered":"2023-07-06T21:42:10.612512+0000","last_clean":"2023-07-06T21:42:10.612512+0000","last_became_active":"2023-07-06T21:42:10.608817+0000","last_became_peered":"2023-07-06T21:42:10.608817+0000","last_unstale":"2023-07-06T21:42:10.612512+0000","last_undegraded":"2023-07-06T21:42:10.612512+0000","last_fullsized":"2023-07-06T21:42:10.612512+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:56.395390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"52'4","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.614153+0000","last_change":"2023-07-06T21:42:10.106103+0000","last_active":"2023-07-06T21:42:10.614153+0000","last_peered":"2023-07-06T21:42:10.614153+0000","last_clean":"2023-07-06T21:42:10.614153+0000","last_became_active":"2023-07-06T21:42:10.105766+0000","last_became_peered":"2023-07-06T21:42:10.105766+0000","last_unstale":"2023-07-06T21:42:10.614153+0000","last_undegraded":"2023-07-06T21:42:10.614153+0000","last_fullsized":"2023-07-06T21:42:10.614153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:50.441575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612832+0000","last_change":"2023-07-06T21:42:10.611656+0000","last_active":"2023-07-06T21:42:10.612832+0000","last_peered":"2023-07-06T21:42:10.612832+0000","last_clean":"2023-07-06T21:42:10.612832+0000","last_became_active":"2023-07-06T21:42:10.611092+0000","last_became_peered":"2023-07-06T21:42:10.611092+0000","last_unstale":"2023-07-06T21:42:10.612832+0000","last_undegraded":"2023-07-06T21:42:10.612832+0000","last_fullsized":"2023-07-06T21:42:10.612832+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:37:33.183332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.613941+0000","last_change":"2023-07-06T21:42:10.109333+0000","last_active":"2023-07-06T21:42:10.613941+0000","last_peered":"2023-07-06T21:42:10.613941+0000","last_clean":"2023-07-06T21:42:10.613941+0000","last_became_active":"2023-07-06T21:42:10.109030+0000","last_became_peered":"2023-07-06T21:42:10.109030+0000","last_unstale":"2023-07-06T21:42:10.613941+0000","last_undegraded":"2023-07-06T21:42:10.613941+0000","last_fullsized":"2023-07-06T21:42:10.613941+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:04:57.920104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612592+0000","last_change":"2023-07-06T21:42:10.603158+0000","last_active":"2023-07-06T21:42:10.612592+0000","last_peered":"2023-07-06T21:42:10.612592+0000","last_clean":"2023-07-06T21:42:10.612592+0000","last_became_active":"2023-07-06T21:42:10.602804+0000","last_became_peered":"2023-07-06T21:42:10.602804+0000","last_unstale":"2023-07-06T21:42:10.612592+0000","last_undegraded":"2023-07-06T21:42:10.612592+0000","last_fullsized":"2023-07-06T21:42:10.612592+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:47:04.939652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612172+0000","last_change":"2023-07-06T21:42:10.106491+0000","last_active":"2023-07-06T21:42:10.612172+0000","last_peered":"2023-07-06T21:42:10.612172+0000","last_clean":"2023-07-06T21:42:10.612172+0000","last_became_active":"2023-07-06T21:42:10.106214+0000","last_became_peered":"2023-07-06T21:42:10.106214+0000","last_unstale":"2023-07-06T21:42:10.612172+0000","last_undegraded":"2023-07-06T21:42:10.612172+0000","last_fullsized":"2023-07-06T21:42:10.612172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:31.046674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612342+0000","last_change":"2023-07-06T21:42:10.605528+0000","last_active":"2023-07-06T21:42:10.612342+0000","last_peered":"2023-07-06T21:42:10.612342+0000","last_clean":"2023-07-06T21:42:10.612342+0000","last_became_active":"2023-07-06T21:42:10.605060+0000","last_became_peered":"2023-07-06T21:42:10.605060+0000","last_unstale":"2023-07-06T21:42:10.612342+0000","last_undegraded":"2023-07-06T21:42:10.612342+0000","last_fullsized":"2023-07-06T21:42:10.612342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:37:33.183332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"52'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591410+0000","last_change":"2023-07-06T21:42:10.105700+0000","last_active":"2023-07-06T21:42:11.591410+0000","last_peered":"2023-07-06T21:42:11.591410+0000","last_clean":"2023-07-06T21:42:11.591410+0000","last_became_active":"2023-07-06T21:42:10.105401+0000","last_became_peered":"2023-07-06T21:42:10.105401+0000","last_unstale":"2023-07-06T21:42:11.591410+0000","last_undegraded":"2023-07-06T21:42:11.591410+0000","last_fullsized":"2023-07-06T21:42:11.591410+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:30:09.460291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606355+0000","last_change":"2023-07-06T21:42:10.598231+0000","last_active":"2023-07-06T21:42:10.606355+0000","last_peered":"2023-07-06T21:42:10.606355+0000","last_clean":"2023-07-06T21:42:10.606355+0000","last_became_active":"2023-07-06T21:42:10.597447+0000","last_became_peered":"2023-07-06T21:42:10.597447+0000","last_unstale":"2023-07-06T21:42:10.606355+0000","last_undegraded":"2023-07-06T21:42:10.606355+0000","last_fullsized":"2023-07-06T21:42:10.606355+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:12:57.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"52'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591408+0000","last_change":"2023-07-06T21:42:10.099044+0000","last_active":"2023-07-06T21:42:11.591408+0000","last_peered":"2023-07-06T21:42:11.591408+0000","last_clean":"2023-07-06T21:42:11.591408+0000","last_became_active":"2023-07-06T21:42:10.098642+0000","last_became_peered":"2023-07-06T21:42:10.098642+0000","last_unstale":"2023-07-06T21:42:11.591408+0000","last_undegraded":"2023-07-06T21:42:11.591408+0000","last_fullsized":"2023-07-06T21:42:11.591408+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:35.393145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610813+0000","last_change":"2023-07-06T21:42:10.603023+0000","last_active":"2023-07-06T21:42:10.610813+0000","last_peered":"2023-07-06T21:42:10.610813+0000","last_clean":"2023-07-06T21:42:10.610813+0000","last_became_active":"2023-07-06T21:42:10.602775+0000","last_became_peered":"2023-07-06T21:42:10.602775+0000","last_unstale":"2023-07-06T21:42:10.610813+0000","last_undegraded":"2023-07-06T21:42:10.610813+0000","last_fullsized":"2023-07-06T21:42:10.610813+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:40:07.262353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.607485+0000","last_change":"2023-07-06T21:42:10.104771+0000","last_active":"2023-07-06T21:42:10.607485+0000","last_peered":"2023-07-06T21:42:10.607485+0000","last_clean":"2023-07-06T21:42:10.607485+0000","last_became_active":"2023-07-06T21:42:10.104426+0000","last_became_peered":"2023-07-06T21:42:10.104426+0000","last_unstale":"2023-07-06T21:42:10.607485+0000","last_undegraded":"2023-07-06T21:42:10.607485+0000","last_fullsized":"2023-07-06T21:42:10.607485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:50.666692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.617406+0000","last_change":"2023-07-06T21:42:10.613088+0000","last_active":"2023-07-06T21:42:10.617406+0000","last_peered":"2023-07-06T21:42:10.617406+0000","last_clean":"2023-07-06T21:42:10.617406+0000","last_became_active":"2023-07-06T21:42:10.612791+0000","last_became_peered":"2023-07-06T21:42:10.612791+0000","last_unstale":"2023-07-06T21:42:10.617406+0000","last_undegraded":"2023-07-06T21:42:10.617406+0000","last_fullsized":"2023-07-06T21:42:10.617406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:10:36.332321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590815+0000","last_change":"2023-07-06T21:42:10.100741+0000","last_active":"2023-07-06T21:42:11.590815+0000","last_peered":"2023-07-06T21:42:11.590815+0000","last_clean":"2023-07-06T21:42:11.590815+0000","last_became_active":"2023-07-06T21:42:10.100453+0000","last_became_peered":"2023-07-06T21:42:10.100453+0000","last_unstale":"2023-07-06T21:42:11.590815+0000","last_undegraded":"2023-07-06T21:42:11.590815+0000","last_fullsized":"2023-07-06T21:42:11.590815+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:22.640052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609719+0000","last_change":"2023-07-06T21:42:10.608496+0000","last_active":"2023-07-06T21:42:10.609719+0000","last_peered":"2023-07-06T21:42:10.609719+0000","last_clean":"2023-07-06T21:42:10.609719+0000","last_became_active":"2023-07-06T21:42:10.608027+0000","last_became_peered":"2023-07-06T21:42:10.608027+0000","last_unstale":"2023-07-06T21:42:10.609719+0000","last_undegraded":"2023-07-06T21:42:10.609719+0000","last_fullsized":"2023-07-06T21:42:10.609719+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:24.653218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590654+0000","last_change":"2023-07-06T21:42:10.106387+0000","last_active":"2023-07-06T21:42:11.590654+0000","last_peered":"2023-07-06T21:42:11.590654+0000","last_clean":"2023-07-06T21:42:11.590654+0000","last_became_active":"2023-07-06T21:42:10.106024+0000","last_became_peered":"2023-07-06T21:42:10.106024+0000","last_unstale":"2023-07-06T21:42:11.590654+0000","last_undegraded":"2023-07-06T21:42:11.590654+0000","last_fullsized":"2023-07-06T21:42:11.590654+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:03.166828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591188+0000","last_change":"2023-07-06T21:42:10.598376+0000","last_active":"2023-07-06T21:42:11.591188+0000","last_peered":"2023-07-06T21:42:11.591188+0000","last_clean":"2023-07-06T21:42:11.591188+0000","last_became_active":"2023-07-06T21:42:10.598075+0000","last_became_peered":"2023-07-06T21:42:10.598075+0000","last_unstale":"2023-07-06T21:42:11.591188+0000","last_undegraded":"2023-07-06T21:42:11.591188+0000","last_fullsized":"2023-07-06T21:42:11.591188+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:25.572938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.599852+0000","last_change":"2023-07-06T21:42:10.107114+0000","last_active":"2023-07-06T21:42:10.599852+0000","last_peered":"2023-07-06T21:42:10.599852+0000","last_clean":"2023-07-06T21:42:10.599852+0000","last_became_active":"2023-07-06T21:42:10.106823+0000","last_became_peered":"2023-07-06T21:42:10.106823+0000","last_unstale":"2023-07-06T21:42:10.599852+0000","last_undegraded":"2023-07-06T21:42:10.599852+0000","last_fullsized":"2023-07-06T21:42:10.599852+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:54:40.407128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619600+0000","last_change":"2023-07-06T21:42:10.612583+0000","last_active":"2023-07-06T21:42:10.619600+0000","last_peered":"2023-07-06T21:42:10.619600+0000","last_clean":"2023-07-06T21:42:10.619600+0000","last_became_active":"2023-07-06T21:42:10.612182+0000","last_became_peered":"2023-07-06T21:42:10.612182+0000","last_unstale":"2023-07-06T21:42:10.619600+0000","last_undegraded":"2023-07-06T21:42:10.619600+0000","last_fullsized":"2023-07-06T21:42:10.619600+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:51:48.928642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.586646+0000","last_change":"2023-07-06T21:42:10.103885+0000","last_active":"2023-07-06T21:42:10.586646+0000","last_peered":"2023-07-06T21:42:10.586646+0000","last_clean":"2023-07-06T21:42:10.586646+0000","last_became_active":"2023-07-06T21:42:10.103485+0000","last_became_peered":"2023-07-06T21:42:10.103485+0000","last_unstale":"2023-07-06T21:42:10.586646+0000","last_undegraded":"2023-07-06T21:42:10.586646+0000","last_fullsized":"2023-07-06T21:42:10.586646+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:58:15.839803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610317+0000","last_change":"2023-07-06T21:42:10.601953+0000","last_active":"2023-07-06T21:42:10.610317+0000","last_peered":"2023-07-06T21:42:10.610317+0000","last_clean":"2023-07-06T21:42:10.610317+0000","last_became_active":"2023-07-06T21:42:10.601683+0000","last_became_peered":"2023-07-06T21:42:10.601683+0000","last_unstale":"2023-07-06T21:42:10.610317+0000","last_undegraded":"2023-07-06T21:42:10.610317+0000","last_fullsized":"2023-07-06T21:42:10.610317+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:14.358591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"52'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.614457+0000","last_change":"2023-07-06T21:42:10.111931+0000","last_active":"2023-07-06T21:42:10.614457+0000","last_peered":"2023-07-06T21:42:10.614457+0000","last_clean":"2023-07-06T21:42:10.614457+0000","last_became_active":"2023-07-06T21:42:10.111673+0000","last_became_peered":"2023-07-06T21:42:10.111673+0000","last_unstale":"2023-07-06T21:42:10.614457+0000","last_undegraded":"2023-07-06T21:42:10.614457+0000","last_fullsized":"2023-07-06T21:42:10.614457+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:33:16.351712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619691+0000","last_change":"2023-07-06T21:42:10.614433+0000","last_active":"2023-07-06T21:42:10.619691+0000","last_peered":"2023-07-06T21:42:10.619691+0000","last_clean":"2023-07-06T21:42:10.619691+0000","last_became_active":"2023-07-06T21:42:10.613807+0000","last_became_peered":"2023-07-06T21:42:10.613807+0000","last_unstale":"2023-07-06T21:42:10.619691+0000","last_undegraded":"2023-07-06T21:42:10.619691+0000","last_fullsized":"2023-07-06T21:42:10.619691+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:27:12.933707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"52'6","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591689+0000","last_change":"2023-07-06T21:42:10.108114+0000","last_active":"2023-07-06T21:42:11.591689+0000","last_peered":"2023-07-06T21:42:11.591689+0000","last_clean":"2023-07-06T21:42:11.591689+0000","last_became_active":"2023-07-06T21:42:10.107799+0000","last_became_peered":"2023-07-06T21:42:10.107799+0000","last_unstale":"2023-07-06T21:42:11.591689+0000","last_undegraded":"2023-07-06T21:42:11.591689+0000","last_fullsized":"2023-07-06T21:42:11.591689+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:38:05.934159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590960+0000","last_change":"2023-07-06T21:42:10.596784+0000","last_active":"2023-07-06T21:42:11.590960+0000","last_peered":"2023-07-06T21:42:11.590960+0000","last_clean":"2023-07-06T21:42:11.590960+0000","last_became_active":"2023-07-06T21:42:10.596479+0000","last_became_peered":"2023-07-06T21:42:10.596479+0000","last_unstale":"2023-07-06T21:42:11.590960+0000","last_undegraded":"2023-07-06T21:42:11.590960+0000","last_fullsized":"2023-07-06T21:42:11.590960+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:48:42.014991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609072+0000","last_change":"2023-07-06T21:42:10.103371+0000","last_active":"2023-07-06T21:42:10.609072+0000","last_peered":"2023-07-06T21:42:10.609072+0000","last_clean":"2023-07-06T21:42:10.609072+0000","last_became_active":"2023-07-06T21:42:10.102943+0000","last_became_peered":"2023-07-06T21:42:10.102943+0000","last_unstale":"2023-07-06T21:42:10.609072+0000","last_undegraded":"2023-07-06T21:42:10.609072+0000","last_fullsized":"2023-07-06T21:42:10.609072+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:29.089916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619483+0000","last_change":"2023-07-06T21:42:10.611021+0000","last_active":"2023-07-06T21:42:10.619483+0000","last_peered":"2023-07-06T21:42:10.619483+0000","last_clean":"2023-07-06T21:42:10.619483+0000","last_became_active":"2023-07-06T21:42:10.610749+0000","last_became_peered":"2023-07-06T21:42:10.610749+0000","last_unstale":"2023-07-06T21:42:10.619483+0000","last_undegraded":"2023-07-06T21:42:10.619483+0000","last_fullsized":"2023-07-06T21:42:10.619483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:00:27.140587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590435+0000","last_change":"2023-07-06T21:42:10.108145+0000","last_active":"2023-07-06T21:42:10.590435+0000","last_peered":"2023-07-06T21:42:10.590435+0000","last_clean":"2023-07-06T21:42:10.590435+0000","last_became_active":"2023-07-06T21:42:10.107833+0000","last_became_peered":"2023-07-06T21:42:10.107833+0000","last_unstale":"2023-07-06T21:42:10.590435+0000","last_undegraded":"2023-07-06T21:42:10.590435+0000","last_fullsized":"2023-07-06T21:42:10.590435+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:33.511306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590510+0000","last_change":"2023-07-06T21:42:10.605169+0000","last_active":"2023-07-06T21:42:11.590510+0000","last_peered":"2023-07-06T21:42:11.590510+0000","last_clean":"2023-07-06T21:42:11.590510+0000","last_became_active":"2023-07-06T21:42:10.604866+0000","last_became_peered":"2023-07-06T21:42:10.604866+0000","last_unstale":"2023-07-06T21:42:11.590510+0000","last_undegraded":"2023-07-06T21:42:11.590510+0000","last_fullsized":"2023-07-06T21:42:11.590510+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:14.024714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590957+0000","last_change":"2023-07-06T21:42:10.103020+0000","last_active":"2023-07-06T21:42:11.590957+0000","last_peered":"2023-07-06T21:42:11.590957+0000","last_clean":"2023-07-06T21:42:11.590957+0000","last_became_active":"2023-07-06T21:42:10.102679+0000","last_became_peered":"2023-07-06T21:42:10.102679+0000","last_unstale":"2023-07-06T21:42:11.590957+0000","last_undegraded":"2023-07-06T21:42:11.590957+0000","last_fullsized":"2023-07-06T21:42:11.590957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:40:29.483560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619466+0000","last_change":"2023-07-06T21:42:10.610661+0000","last_active":"2023-07-06T21:42:10.619466+0000","last_peered":"2023-07-06T21:42:10.619466+0000","last_clean":"2023-07-06T21:42:10.619466+0000","last_became_active":"2023-07-06T21:42:10.609061+0000","last_became_peered":"2023-07-06T21:42:10.609061+0000","last_unstale":"2023-07-06T21:42:10.619466+0000","last_undegraded":"2023-07-06T21:42:10.619466+0000","last_fullsized":"2023-07-06T21:42:10.619466+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:12.543043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"52'8","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.616638+0000","last_change":"2023-07-06T21:42:10.109338+0000","last_active":"2023-07-06T21:42:10.616638+0000","last_peered":"2023-07-06T21:42:10.616638+0000","last_clean":"2023-07-06T21:42:10.616638+0000","last_became_active":"2023-07-06T21:42:10.109020+0000","last_became_peered":"2023-07-06T21:42:10.109020+0000","last_unstale":"2023-07-06T21:42:10.616638+0000","last_undegraded":"2023-07-06T21:42:10.616638+0000","last_fullsized":"2023-07-06T21:42:10.616638+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:17.346674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591597+0000","last_change":"2023-07-06T21:42:10.600611+0000","last_active":"2023-07-06T21:42:11.591597+0000","last_peered":"2023-07-06T21:42:11.591597+0000","last_clean":"2023-07-06T21:42:11.591597+0000","last_became_active":"2023-07-06T21:42:10.600106+0000","last_became_peered":"2023-07-06T21:42:10.600106+0000","last_unstale":"2023-07-06T21:42:11.591597+0000","last_undegraded":"2023-07-06T21:42:11.591597+0000","last_fullsized":"2023-07-06T21:42:11.591597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:18.746595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610189+0000","last_change":"2023-07-06T21:42:10.109503+0000","last_active":"2023-07-06T21:42:10.610189+0000","last_peered":"2023-07-06T21:42:10.610189+0000","last_clean":"2023-07-06T21:42:10.610189+0000","last_became_active":"2023-07-06T21:42:10.109208+0000","last_became_peered":"2023-07-06T21:42:10.109208+0000","last_unstale":"2023-07-06T21:42:10.610189+0000","last_undegraded":"2023-07-06T21:42:10.610189+0000","last_fullsized":"2023-07-06T21:42:10.610189+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:05:07.616090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591659+0000","last_change":"2023-07-06T21:42:10.615022+0000","last_active":"2023-07-06T21:42:11.591659+0000","last_peered":"2023-07-06T21:42:11.591659+0000","last_clean":"2023-07-06T21:42:11.591659+0000","last_became_active":"2023-07-06T21:42:10.614673+0000","last_became_peered":"2023-07-06T21:42:10.614673+0000","last_unstale":"2023-07-06T21:42:11.591659+0000","last_undegraded":"2023-07-06T21:42:11.591659+0000","last_fullsized":"2023-07-06T21:42:11.591659+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:47:42.190957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590153+0000","last_change":"2023-07-06T21:42:10.106050+0000","last_active":"2023-07-06T21:42:10.590153+0000","last_peered":"2023-07-06T21:42:10.590153+0000","last_clean":"2023-07-06T21:42:10.590153+0000","last_became_active":"2023-07-06T21:42:10.105796+0000","last_became_peered":"2023-07-06T21:42:10.105796+0000","last_unstale":"2023-07-06T21:42:10.590153+0000","last_undegraded":"2023-07-06T21:42:10.590153+0000","last_fullsized":"2023-07-06T21:42:10.590153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:38.363754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.611233+0000","last_change":"2023-07-06T21:42:10.604268+0000","last_active":"2023-07-06T21:42:10.611233+0000","last_peered":"2023-07-06T21:42:10.611233+0000","last_clean":"2023-07-06T21:42:10.611233+0000","last_became_active":"2023-07-06T21:42:10.603968+0000","last_became_peered":"2023-07-06T21:42:10.603968+0000","last_unstale":"2023-07-06T21:42:10.611233+0000","last_undegraded":"2023-07-06T21:42:10.611233+0000","last_fullsized":"2023-07-06T21:42:10.611233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:47:04.939652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590195+0000","last_change":"2023-07-06T21:42:10.101939+0000","last_active":"2023-07-06T21:42:10.590195+0000","last_peered":"2023-07-06T21:42:10.590195+0000","last_clean":"2023-07-06T21:42:10.590195+0000","last_became_active":"2023-07-06T21:42:10.101645+0000","last_became_peered":"2023-07-06T21:42:10.101645+0000","last_unstale":"2023-07-06T21:42:10.590195+0000","last_undegraded":"2023-07-06T21:42:10.590195+0000","last_fullsized":"2023-07-06T21:42:10.590195+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:16:48.900805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612322+0000","last_change":"2023-07-06T21:42:10.605303+0000","last_active":"2023-07-06T21:42:10.612322+0000","last_peered":"2023-07-06T21:42:10.612322+0000","last_clean":"2023-07-06T21:42:10.612322+0000","last_became_active":"2023-07-06T21:42:10.604761+0000","last_became_peered":"2023-07-06T21:42:10.604761+0000","last_unstale":"2023-07-06T21:42:10.612322+0000","last_undegraded":"2023-07-06T21:42:10.612322+0000","last_fullsized":"2023-07-06T21:42:10.612322+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:22:21.583971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591059+0000","last_change":"2023-07-06T21:42:10.100770+0000","last_active":"2023-07-06T21:42:11.591059+0000","last_peered":"2023-07-06T21:42:11.591059+0000","last_clean":"2023-07-06T21:42:11.591059+0000","last_became_active":"2023-07-06T21:42:10.100490+0000","last_became_peered":"2023-07-06T21:42:10.100490+0000","last_unstale":"2023-07-06T21:42:11.591059+0000","last_undegraded":"2023-07-06T21:42:11.591059+0000","last_fullsized":"2023-07-06T21:42:11.591059+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:33:01.646226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.596509+0000","last_change":"2023-07-06T21:42:10.594392+0000","last_active":"2023-07-06T21:42:10.596509+0000","last_peered":"2023-07-06T21:42:10.596509+0000","last_clean":"2023-07-06T21:42:10.596509+0000","last_became_active":"2023-07-06T21:42:10.594141+0000","last_became_peered":"2023-07-06T21:42:10.594141+0000","last_unstale":"2023-07-06T21:42:10.596509+0000","last_undegraded":"2023-07-06T21:42:10.596509+0000","last_fullsized":"2023-07-06T21:42:10.596509+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:04.176809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.588657+0000","last_change":"2023-07-06T21:42:10.109157+0000","last_active":"2023-07-06T21:42:10.588657+0000","last_peered":"2023-07-06T21:42:10.588657+0000","last_clean":"2023-07-06T21:42:10.588657+0000","last_became_active":"2023-07-06T21:42:10.108868+0000","last_became_peered":"2023-07-06T21:42:10.108868+0000","last_unstale":"2023-07-06T21:42:10.588657+0000","last_undegraded":"2023-07-06T21:42:10.588657+0000","last_fullsized":"2023-07-06T21:42:10.588657+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:47.457572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619543+0000","last_change":"2023-07-06T21:42:10.611408+0000","last_active":"2023-07-06T21:42:10.619543+0000","last_peered":"2023-07-06T21:42:10.619543+0000","last_clean":"2023-07-06T21:42:10.619543+0000","last_became_active":"2023-07-06T21:42:10.610951+0000","last_became_peered":"2023-07-06T21:42:10.610951+0000","last_unstale":"2023-07-06T21:42:10.619543+0000","last_undegraded":"2023-07-06T21:42:10.619543+0000","last_fullsized":"2023-07-06T21:42:10.619543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:46:45.859395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591153+0000","last_change":"2023-07-06T21:42:10.106381+0000","last_active":"2023-07-06T21:42:11.591153+0000","last_peered":"2023-07-06T21:42:11.591153+0000","last_clean":"2023-07-06T21:42:11.591153+0000","last_became_active":"2023-07-06T21:42:10.106075+0000","last_became_peered":"2023-07-06T21:42:10.106075+0000","last_unstale":"2023-07-06T21:42:11.591153+0000","last_undegraded":"2023-07-06T21:42:11.591153+0000","last_fullsized":"2023-07-06T21:42:11.591153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:12:56.499349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612038+0000","last_change":"2023-07-06T21:42:10.605265+0000","last_active":"2023-07-06T21:42:10.612038+0000","last_peered":"2023-07-06T21:42:10.612038+0000","last_clean":"2023-07-06T21:42:10.612038+0000","last_became_active":"2023-07-06T21:42:10.604930+0000","last_became_peered":"2023-07-06T21:42:10.604930+0000","last_unstale":"2023-07-06T21:42:10.612038+0000","last_undegraded":"2023-07-06T21:42:10.612038+0000","last_fullsized":"2023-07-06T21:42:10.612038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:39.373736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.586609+0000","last_change":"2023-07-06T21:42:10.107339+0000","last_active":"2023-07-06T21:42:10.586609+0000","last_peered":"2023-07-06T21:42:10.586609+0000","last_clean":"2023-07-06T21:42:10.586609+0000","last_became_active":"2023-07-06T21:42:10.107061+0000","last_became_peered":"2023-07-06T21:42:10.107061+0000","last_unstale":"2023-07-06T21:42:10.586609+0000","last_undegraded":"2023-07-06T21:42:10.586609+0000","last_fullsized":"2023-07-06T21:42:10.586609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:55.385408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609699+0000","last_change":"2023-07-06T21:42:10.608287+0000","last_active":"2023-07-06T21:42:10.609699+0000","last_peered":"2023-07-06T21:42:10.609699+0000","last_clean":"2023-07-06T21:42:10.609699+0000","last_became_active":"2023-07-06T21:42:10.607617+0000","last_became_peered":"2023-07-06T21:42:10.607617+0000","last_unstale":"2023-07-06T21:42:10.609699+0000","last_undegraded":"2023-07-06T21:42:10.609699+0000","last_fullsized":"2023-07-06T21:42:10.609699+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:49:05.964671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.592796+0000","last_change":"2023-07-06T21:42:10.098590+0000","last_active":"2023-07-06T21:42:10.592796+0000","last_peered":"2023-07-06T21:42:10.592796+0000","last_clean":"2023-07-06T21:42:10.592796+0000","last_became_active":"2023-07-06T21:42:10.098111+0000","last_became_peered":"2023-07-06T21:42:10.098111+0000","last_unstale":"2023-07-06T21:42:10.592796+0000","last_undegraded":"2023-07-06T21:42:10.592796+0000","last_fullsized":"2023-07-06T21:42:10.592796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:00:24.466871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591603+0000","last_change":"2023-07-06T21:42:10.601307+0000","last_active":"2023-07-06T21:42:11.591603+0000","last_peered":"2023-07-06T21:42:11.591603+0000","last_clean":"2023-07-06T21:42:11.591603+0000","last_became_active":"2023-07-06T21:42:10.600999+0000","last_became_peered":"2023-07-06T21:42:10.600999+0000","last_unstale":"2023-07-06T21:42:11.591603+0000","last_undegraded":"2023-07-06T21:42:11.591603+0000","last_fullsized":"2023-07-06T21:42:11.591603+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:13:18.541193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.607379+0000","last_change":"2023-07-06T21:42:10.106022+0000","last_active":"2023-07-06T21:42:10.607379+0000","last_peered":"2023-07-06T21:42:10.607379+0000","last_clean":"2023-07-06T21:42:10.607379+0000","last_became_active":"2023-07-06T21:42:10.105743+0000","last_became_peered":"2023-07-06T21:42:10.105743+0000","last_unstale":"2023-07-06T21:42:10.607379+0000","last_undegraded":"2023-07-06T21:42:10.607379+0000","last_fullsized":"2023-07-06T21:42:10.607379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:13.348609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.587414+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:42:10.587414+0000","last_peered":"2023-07-06T21:42:10.587414+0000","last_clean":"2023-07-06T21:42:10.587414+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:42:10.587414+0000","last_undegraded":"2023-07-06T21:42:10.587414+0000","last_fullsized":"2023-07-06T21:42:10.587414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:15:26.222928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612369+0000","last_change":"2023-07-06T21:42:10.607093+0000","last_active":"2023-07-06T21:42:10.612369+0000","last_peered":"2023-07-06T21:42:10.612369+0000","last_clean":"2023-07-06T21:42:10.612369+0000","last_became_active":"2023-07-06T21:42:10.603592+0000","last_became_peered":"2023-07-06T21:42:10.603592+0000","last_unstale":"2023-07-06T21:42:10.612369+0000","last_undegraded":"2023-07-06T21:42:10.612369+0000","last_fullsized":"2023-07-06T21:42:10.612369+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:04:09.902720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590787+0000","last_change":"2023-07-06T21:42:10.103313+0000","last_active":"2023-07-06T21:42:11.590787+0000","last_peered":"2023-07-06T21:42:11.590787+0000","last_clean":"2023-07-06T21:42:11.590787+0000","last_became_active":"2023-07-06T21:42:10.102646+0000","last_became_peered":"2023-07-06T21:42:10.102646+0000","last_unstale":"2023-07-06T21:42:11.590787+0000","last_undegraded":"2023-07-06T21:42:11.590787+0000","last_fullsized":"2023-07-06T21:42:11.590787+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:19:30.095342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591063+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:42:11.591063+0000","last_peered":"2023-07-06T21:42:11.591063+0000","last_clean":"2023-07-06T21:42:11.591063+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:42:11.591063+0000","last_undegraded":"2023-07-06T21:42:11.591063+0000","last_fullsized":"2023-07-06T21:42:11.591063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:35.603909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591278+0000","last_change":"2023-07-06T21:42:10.602481+0000","last_active":"2023-07-06T21:42:11.591278+0000","last_peered":"2023-07-06T21:42:11.591278+0000","last_clean":"2023-07-06T21:42:11.591278+0000","last_became_active":"2023-07-06T21:42:10.602127+0000","last_became_peered":"2023-07-06T21:42:10.602127+0000","last_unstale":"2023-07-06T21:42:11.591278+0000","last_undegraded":"2023-07-06T21:42:11.591278+0000","last_fullsized":"2023-07-06T21:42:11.591278+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:54:23.563056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610545+0000","last_change":"2023-07-06T21:42:10.103206+0000","last_active":"2023-07-06T21:42:10.610545+0000","last_peered":"2023-07-06T21:42:10.610545+0000","last_clean":"2023-07-06T21:42:10.610545+0000","last_became_active":"2023-07-06T21:42:10.102654+0000","last_became_peered":"2023-07-06T21:42:10.102654+0000","last_unstale":"2023-07-06T21:42:10.610545+0000","last_undegraded":"2023-07-06T21:42:10.610545+0000","last_fullsized":"2023-07-06T21:42:10.610545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:24.562956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.588854+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:42:11.588854+0000","last_peered":"2023-07-06T21:42:11.588854+0000","last_clean":"2023-07-06T21:42:11.588854+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:42:11.588854+0000","last_undegraded":"2023-07-06T21:42:11.588854+0000","last_fullsized":"2023-07-06T21:42:11.588854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:08.531361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.608624+0000","last_change":"2023-07-06T21:42:10.602370+0000","last_active":"2023-07-06T21:42:10.608624+0000","last_peered":"2023-07-06T21:42:10.608624+0000","last_clean":"2023-07-06T21:42:10.608624+0000","last_became_active":"2023-07-06T21:42:10.602011+0000","last_became_peered":"2023-07-06T21:42:10.602011+0000","last_unstale":"2023-07-06T21:42:10.608624+0000","last_undegraded":"2023-07-06T21:42:10.608624+0000","last_fullsized":"2023-07-06T21:42:10.608624+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:36.403127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.595197+0000","last_change":"2023-07-06T21:42:10.106213+0000","last_active":"2023-07-06T21:42:10.595197+0000","last_peered":"2023-07-06T21:42:10.595197+0000","last_clean":"2023-07-06T21:42:10.595197+0000","last_became_active":"2023-07-06T21:42:10.105939+0000","last_became_peered":"2023-07-06T21:42:10.105939+0000","last_unstale":"2023-07-06T21:42:10.595197+0000","last_undegraded":"2023-07-06T21:42:10.595197+0000","last_fullsized":"2023-07-06T21:42:10.595197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:10:17.223996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590846+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:42:11.590846+0000","last_peered":"2023-07-06T21:42:11.590846+0000","last_clean":"2023-07-06T21:42:11.590846+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:42:11.590846+0000","last_undegraded":"2023-07-06T21:42:11.590846+0000","last_fullsized":"2023-07-06T21:42:11.590846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:49:13.828393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.598075+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:42:10.598075+0000","last_peered":"2023-07-06T21:42:10.598075+0000","last_clean":"2023-07-06T21:42:10.598075+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:42:10.598075+0000","last_undegraded":"2023-07-06T21:42:10.598075+0000","last_fullsized":"2023-07-06T21:42:10.598075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:02.236521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.611714+0000","last_change":"2023-07-06T21:42:10.602778+0000","last_active":"2023-07-06T21:42:10.611714+0000","last_peered":"2023-07-06T21:42:10.611714+0000","last_clean":"2023-07-06T21:42:10.611714+0000","last_became_active":"2023-07-06T21:42:10.602495+0000","last_became_peered":"2023-07-06T21:42:10.602495+0000","last_unstale":"2023-07-06T21:42:10.611714+0000","last_undegraded":"2023-07-06T21:42:10.611714+0000","last_fullsized":"2023-07-06T21:42:10.611714+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:01:12.971613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.587950+0000","last_change":"2023-07-06T21:42:10.106963+0000","last_active":"2023-07-06T21:42:10.587950+0000","last_peered":"2023-07-06T21:42:10.587950+0000","last_clean":"2023-07-06T21:42:10.587950+0000","last_became_active":"2023-07-06T21:42:10.106586+0000","last_became_peered":"2023-07-06T21:42:10.106586+0000","last_unstale":"2023-07-06T21:42:10.587950+0000","last_undegraded":"2023-07-06T21:42:10.587950+0000","last_fullsized":"2023-07-06T21:42:10.587950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:50.666692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.597945+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:42:10.597945+0000","last_peered":"2023-07-06T21:42:10.597945+0000","last_clean":"2023-07-06T21:42:10.597945+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:42:10.597945+0000","last_undegraded":"2023-07-06T21:42:10.597945+0000","last_fullsized":"2023-07-06T21:42:10.597945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:14:21.065092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619564+0000","last_change":"2023-07-06T21:42:10.611525+0000","last_active":"2023-07-06T21:42:10.619564+0000","last_peered":"2023-07-06T21:42:10.619564+0000","last_clean":"2023-07-06T21:42:10.619564+0000","last_became_active":"2023-07-06T21:42:10.611211+0000","last_became_peered":"2023-07-06T21:42:10.611211+0000","last_unstale":"2023-07-06T21:42:10.619564+0000","last_undegraded":"2023-07-06T21:42:10.619564+0000","last_fullsized":"2023-07-06T21:42:10.619564+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:45:15.650121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.607722+0000","last_change":"2023-07-06T21:42:10.108755+0000","last_active":"2023-07-06T21:42:10.607722+0000","last_peered":"2023-07-06T21:42:10.607722+0000","last_clean":"2023-07-06T21:42:10.607722+0000","last_became_active":"2023-07-06T21:42:10.108439+0000","last_became_peered":"2023-07-06T21:42:10.108439+0000","last_unstale":"2023-07-06T21:42:10.607722+0000","last_undegraded":"2023-07-06T21:42:10.607722+0000","last_fullsized":"2023-07-06T21:42:10.607722+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:47:12.139600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.601337+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:42:10.601337+0000","last_peered":"2023-07-06T21:42:10.601337+0000","last_clean":"2023-07-06T21:42:10.601337+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:42:10.601337+0000","last_undegraded":"2023-07-06T21:42:10.601337+0000","last_fullsized":"2023-07-06T21:42:10.601337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:17:35.720425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619671+0000","last_change":"2023-07-06T21:42:10.614243+0000","last_active":"2023-07-06T21:42:10.619671+0000","last_peered":"2023-07-06T21:42:10.619671+0000","last_clean":"2023-07-06T21:42:10.619671+0000","last_became_active":"2023-07-06T21:42:10.613762+0000","last_became_peered":"2023-07-06T21:42:10.613762+0000","last_unstale":"2023-07-06T21:42:10.619671+0000","last_undegraded":"2023-07-06T21:42:10.619671+0000","last_fullsized":"2023-07-06T21:42:10.619671+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:57:25.798752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590102+0000","last_change":"2023-07-06T21:42:10.107574+0000","last_active":"2023-07-06T21:42:10.590102+0000","last_peered":"2023-07-06T21:42:10.590102+0000","last_clean":"2023-07-06T21:42:10.590102+0000","last_became_active":"2023-07-06T21:42:10.107320+0000","last_became_peered":"2023-07-06T21:42:10.107320+0000","last_unstale":"2023-07-06T21:42:10.590102+0000","last_undegraded":"2023-07-06T21:42:10.590102+0000","last_fullsized":"2023-07-06T21:42:10.590102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:15.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.589210+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:42:10.589210+0000","last_peered":"2023-07-06T21:42:10.589210+0000","last_clean":"2023-07-06T21:42:10.589210+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:42:10.589210+0000","last_undegraded":"2023-07-06T21:42:10.589210+0000","last_fullsized":"2023-07-06T21:42:10.589210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:16.679462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606646+0000","last_change":"2023-07-06T21:42:10.600739+0000","last_active":"2023-07-06T21:42:10.606646+0000","last_peered":"2023-07-06T21:42:10.606646+0000","last_clean":"2023-07-06T21:42:10.606646+0000","last_became_active":"2023-07-06T21:42:10.600485+0000","last_became_peered":"2023-07-06T21:42:10.600485+0000","last_unstale":"2023-07-06T21:42:10.606646+0000","last_undegraded":"2023-07-06T21:42:10.606646+0000","last_fullsized":"2023-07-06T21:42:10.606646+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:10.542591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"52'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591493+0000","last_change":"2023-07-06T21:42:10.108778+0000","last_active":"2023-07-06T21:42:11.591493+0000","last_peered":"2023-07-06T21:42:11.591493+0000","last_clean":"2023-07-06T21:42:11.591493+0000","last_became_active":"2023-07-06T21:42:10.108395+0000","last_became_peered":"2023-07-06T21:42:10.108395+0000","last_unstale":"2023-07-06T21:42:11.591493+0000","last_undegraded":"2023-07-06T21:42:11.591493+0000","last_fullsized":"2023-07-06T21:42:11.591493+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:29:24.120723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.591120+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:42:10.591120+0000","last_peered":"2023-07-06T21:42:10.591120+0000","last_clean":"2023-07-06T21:42:10.591120+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:42:10.591120+0000","last_undegraded":"2023-07-06T21:42:10.591120+0000","last_fullsized":"2023-07-06T21:42:10.591120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:09:00.212444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1220608,"data_stored":860226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85999999999999999,"5min":1.28,"15min":1.28},"last":1.101},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":0.995,"15min":0.995},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.43099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.042},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82599999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93200000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83099999999999996,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.85299999999999998,"5min":1.325,"15min":1.325},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.73899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86599999999999999,"5min":1.972,"15min":1.972},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.379,"15min":1.379},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97599999999999998,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.996,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":618496,"data_stored":269698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.21,"5min":3.569,"15min":3.569},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.50900000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79200000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.121,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.497}]},{"osd":2,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.79000000000000004,"15min":0.81399999999999995},"min":{"1min":0.441,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.014,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79100000000000004,"15min":0.81100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92100000000000004,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.032,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.83999999999999997,"15min":0.86499999999999999},"min":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98499999999999999,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.40000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.871,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":622592,"data_stored":269788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.63700000000000001,"15min":0.65700000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":3.081,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6200000000000001,"5min":2.246,"15min":2.246},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.56599999999999995,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96499999999999997,"15min":0.97799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":9.6300000000000008,"5min":11.913,"15min":11.913},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":1.262,"15min":1.3600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.4489999999999998,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.89700000000000002,"15min":0.95099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8370000000000002,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80000000000000004,"15min":0.82099999999999995},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.2649999999999997,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.9729999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.871},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95899999999999996,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90500000000000003,"15min":0.90900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5540000000000003,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.7110000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.85599999999999998,"15min":0.86799999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.984,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.92100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1224704,"data_stored":861193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.51400000000000001,"15min":0.53800000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54000000000000004,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.52100000000000002,"15min":0.53700000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65100000000000002,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.57799999999999996,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58999999999999997,"5min":2.29,"15min":2.29},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70299999999999996,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60799999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.498}]},{"osd":4,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.83699999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90300000000000002,"5min":2.234,"15min":2.234},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.84999999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86599999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.82999999999999996,"15min":0.85599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.82799999999999996,"5min":3.544,"15min":3.544},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82799999999999996},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":2.367,"15min":2.367},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.84399999999999997,"15min":0.86599999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92200000000000004,"5min":2.395,"15min":2.395},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.83099999999999996,"15min":0.85499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91200000000000003,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.874,"15min":0.90100000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.89100000000000001,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":0.80800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27400000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.435,"15min":0.43099999999999999},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.316,"5min":0.315,"15min":0.315},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61599999999999999,"5min":1.369,"15min":1.369},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.61799999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.57399999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.435}]},{"osd":4,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.86499999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.746},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.59299999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83299999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.878,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.86599999999999999,"5min":1.476,"15min":1.476},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.86699999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.629},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91800000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.83499999999999996,"15min":0.79700000000000004},"min":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":8.3190000000000008,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.95999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":269754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.432,"15min":0.434},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.309},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.437,"15min":0.442},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59099999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.44}]},{"osd":1,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47599999999999998,"15min":0.47799999999999998},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.61399999999999999,"5min":1.155,"15min":1.155},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88200000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.51700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.304,"15min":1.304},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":1.258,"15min":1.258},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.81200000000000006,"5min":2.012,"15min":2.012},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.53800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93100000000000005,"5min":1.29,"15min":1.29},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.77700000000000002,"15min":0.79000000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.875,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":1.121,"15min":1.121},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":643072,"data_stored":277915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.621},"min":{"1min":0.45900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.53900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.498,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.431,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.75700000000000001},"min":{"1min":0.44,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.439,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.46600000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91700000000000004,"5min":1.46,"15min":1.46},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78200000000000003,"15min":0.77900000000000003},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":270350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95299999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.498},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.876,"5min":1.038,"15min":1.038},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.90300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.628}]},{"osd":3,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94199999999999995,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84499999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84999999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":0.89600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.751,"15min":0.75700000000000001},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85799999999999998,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77700000000000002,"15min":0.78000000000000003},"min":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90200000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.96899999999999997,"5min":1.278,"15min":1.278},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:42:14.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:42:15.060 INFO:tasks.ceph.mds.d.smithi107.stderr:2023-07-06T21:42:15.058+0000 7f208c1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T21:42:15.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.180+0000 7f9fc89ff700 1 Processor -- start 2023-07-06T21:42:15.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.180+0000 7f9fc89ff700 1 -- start start 2023-07-06T21:42:15.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.181+0000 7f9fc89ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fc40d3710 0x7f9fc40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.181+0000 7f9fc89ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40cd910 con 0x7f9fc40cfa90 2023-07-06T21:42:15.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fc89ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40d0a80 con 0x7f9fc40b83c0 2023-07-06T21:42:15.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fc89ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc40cff10 con 0x7f9fc40d3710 2023-07-06T21:42:15.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fc40d3710 0x7f9fc40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9fc40d3710 0x7f9fc40cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57334/0 (socket says 172.21.15.84:57334) 2023-07-06T21:42:15.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fc2d9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9fc40cfa90 0x7f9fc40d0f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56194/0 (socket says 172.21.15.84:56194) 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 -- 172.21.15.84:0/3919928212 learned_addr learned my addr 172.21.15.84:0/3919928212 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 -- 172.21.15.84:0/3919928212 >> v1:172.21.15.107:6789/0 conn(0x7f9fc40b83c0 legacy=0x7f9fc40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 -- 172.21.15.84:0/3919928212 >> v1:172.21.15.84:6789/0 conn(0x7f9fc40cfa90 legacy=0x7f9fc40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 -- 172.21.15.84:0/3919928212 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40b8790 con 0x7f9fc40d3710 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fc1d9b700 1 -- 172.21.15.84:0/3919928212 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426797873 0 0) 0x7f9fbc001490 con 0x7f9fc40cfa90 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.182+0000 7f9fbbfff700 1 --2- 172.21.15.84:0/3919928212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40cf550 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0098f0 tx=0x7f9fac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980b51a13e0da4b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:15.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc1d9b700 1 -- 172.21.15.84:0/3919928212 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00b8a0 con 0x7f9fc40d3710 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc1d9b700 1 -- 172.21.15.84:0/3919928212 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fac00ba00 con 0x7f9fc40d3710 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc1d9b700 1 -- 172.21.15.84:0/3919928212 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00bd90 con 0x7f9fc40d3710 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 -- 172.21.15.84:0/3919928212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 msgr2=0x7f9fc40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/3919928212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40cf550 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0098f0 tx=0x7f9fac009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 -- 172.21.15.84:0/3919928212 shutdown_connections 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/3919928212 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40cf550 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 -- 172.21.15.84:0/3919928212 >> 172.21.15.84:0/3919928212 conn(0x7f9fc406b350 msgr2=0x7f9fc406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:15.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 -- 172.21.15.84:0/3919928212 shutdown_connections 2023-07-06T21:42:15.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.183+0000 7f9fc89ff700 1 -- 172.21.15.84:0/3919928212 wait complete. 2023-07-06T21:42:15.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.186+0000 7f9fc89ff700 1 Processor -- start 2023-07-06T21:42:15.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.186+0000 7f9fc89ff700 1 -- start start 2023-07-06T21:42:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc89ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fc40b83c0 0x7f9fc415de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc89ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fc40cfa90 0x7f9fc40b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc89ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc89ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc415cb40 con 0x7f9fc40d3710 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc89ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc415d9a0 con 0x7f9fc40b83c0 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc89ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc415cd90 con 0x7f9fc40cfa90 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.187+0000 7f9fc2d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40b36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57346/0 (socket says 172.21.15.84:57346) 2023-07-06T21:42:15.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 -- 172.21.15.84:0/494340116 learned_addr learned my addr 172.21.15.84:0/494340116 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:15.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc259c700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fc40b83c0 0x7f9fc415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fbbfff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fc40cfa90 0x7f9fc40b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 -- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fc40cfa90 msgr2=0x7f9fc40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fc40cfa90 0x7f9fc40b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 -- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fc40b83c0 msgr2=0x7f9fc415de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fc40b83c0 0x7f9fc415de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 -- 172.21.15.84:0/494340116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac009550 con 0x7f9fc40d3710 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc2d9d700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40b36c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00cba0 tx=0x7f9fbc00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc00aaa0 con 0x7f9fc40d3710 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fbc00ac20 con 0x7f9fc40d3710 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0070d0 con 0x7f9fc40d3710 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.188+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc415d070 con 0x7f9fc40d3710 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.189+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc415d630 con 0x7f9fc40d3710 2023-07-06T21:42:15.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.190+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc40ce760 con 0x7f9fc40d3710 2023-07-06T21:42:15.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.191+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9fbc007230 con 0x7f9fc40d3710 2023-07-06T21:42:15.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.191+0000 7f9fbaffd700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fa406ace0 0x7f9fa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.192+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fbc088e30 con 0x7f9fc40d3710 2023-07-06T21:42:15.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.192+0000 7f9fc259c700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fa406ace0 0x7f9fa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.192+0000 7f9fc259c700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fa406ace0 0x7f9fa406d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4007fd0 tx=0x7f9fb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:15.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.196+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9fbc053e40 con 0x7f9fc40d3710 2023-07-06T21:42:15.389 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.387+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9fc40b5150 con 0x7f9fa406ace0 2023-07-06T21:42:15.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.395+0000 7f9fbaffd700 1 -- 172.21.15.84:0/494340116 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311283 (secure 0 0 0) 0x7f9fb4001cc0 con 0x7f9fa406ace0 2023-07-06T21:42:15.400 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fa406ace0 msgr2=0x7f9fa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fa406ace0 0x7f9fa406d190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4007fd0 tx=0x7f9fb4009040 comp rx=0 tx=0).stop 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 msgr2=0x7f9fc40b36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40b36c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00cba0 tx=0x7f9fbc00ceb0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 shutdown_connections 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fa406ace0 0x7f9fa406d190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9fc40d3710 0x7f9fc40b36c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fc40b83c0 0x7f9fc415de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 --2- 172.21.15.84:0/494340116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fc40cfa90 0x7f9fc40b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 >> 172.21.15.84:0/494340116 conn(0x7f9fc406b350 msgr2=0x7f9fc40b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.399+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 shutdown_connections 2023-07-06T21:42:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.400+0000 7f9fc89ff700 1 -- 172.21.15.84:0/494340116 wait complete. 2023-07-06T21:42:15.404 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:42:15.420 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2023-07-06T21:42:14.170767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221692,"kb_used_data":6076,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754753028,"statfs":{"total":773094113280,"available":772867100672,"internally_reserved":0,"allocated":6221824,"data_stored":3355331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.087877"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589343+0000","last_change":"2023-07-06T21:42:10.595808+0000","last_active":"2023-07-06T21:42:11.589343+0000","last_peered":"2023-07-06T21:42:11.589343+0000","last_clean":"2023-07-06T21:42:11.589343+0000","last_became_active":"2023-07-06T21:42:10.595536+0000","last_became_peered":"2023-07-06T21:42:10.595536+0000","last_unstale":"2023-07-06T21:42:11.589343+0000","last_undegraded":"2023-07-06T21:42:11.589343+0000","last_fullsized":"2023-07-06T21:42:11.589343+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:17:07.073382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.599197+0000","last_change":"2023-07-06T21:42:10.594349+0000","last_active":"2023-07-06T21:42:10.599197+0000","last_peered":"2023-07-06T21:42:10.599197+0000","last_clean":"2023-07-06T21:42:10.599197+0000","last_became_active":"2023-07-06T21:42:10.594082+0000","last_became_peered":"2023-07-06T21:42:10.594082+0000","last_unstale":"2023-07-06T21:42:10.599197+0000","last_undegraded":"2023-07-06T21:42:10.599197+0000","last_fullsized":"2023-07-06T21:42:10.599197+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:59:59.104906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619441+0000","last_change":"2023-07-06T21:42:10.610344+0000","last_active":"2023-07-06T21:42:10.619441+0000","last_peered":"2023-07-06T21:42:10.619441+0000","last_clean":"2023-07-06T21:42:10.619441+0000","last_became_active":"2023-07-06T21:42:10.608807+0000","last_became_peered":"2023-07-06T21:42:10.608807+0000","last_unstale":"2023-07-06T21:42:10.619441+0000","last_undegraded":"2023-07-06T21:42:10.619441+0000","last_fullsized":"2023-07-06T21:42:10.619441+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:37:03.501526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590759+0000","last_change":"2023-07-06T21:42:10.601610+0000","last_active":"2023-07-06T21:42:11.590759+0000","last_peered":"2023-07-06T21:42:11.590759+0000","last_clean":"2023-07-06T21:42:11.590759+0000","last_became_active":"2023-07-06T21:42:10.601222+0000","last_became_peered":"2023-07-06T21:42:10.601222+0000","last_unstale":"2023-07-06T21:42:11.590759+0000","last_undegraded":"2023-07-06T21:42:11.590759+0000","last_fullsized":"2023-07-06T21:42:11.590759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:13:15.832150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589160+0000","last_change":"2023-07-06T21:42:10.605718+0000","last_active":"2023-07-06T21:42:11.589160+0000","last_peered":"2023-07-06T21:42:11.589160+0000","last_clean":"2023-07-06T21:42:11.589160+0000","last_became_active":"2023-07-06T21:42:10.605414+0000","last_became_peered":"2023-07-06T21:42:10.605414+0000","last_unstale":"2023-07-06T21:42:11.589160+0000","last_undegraded":"2023-07-06T21:42:11.589160+0000","last_fullsized":"2023-07-06T21:42:11.589160+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:14:27.518098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.601551+0000","last_change":"2023-07-06T21:42:10.598487+0000","last_active":"2023-07-06T21:42:10.601551+0000","last_peered":"2023-07-06T21:42:10.601551+0000","last_clean":"2023-07-06T21:42:10.601551+0000","last_became_active":"2023-07-06T21:42:10.598234+0000","last_became_peered":"2023-07-06T21:42:10.598234+0000","last_unstale":"2023-07-06T21:42:10.601551+0000","last_undegraded":"2023-07-06T21:42:10.601551+0000","last_fullsized":"2023-07-06T21:42:10.601551+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:44:30.629246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590827+0000","last_change":"2023-07-06T21:42:10.604054+0000","last_active":"2023-07-06T21:42:11.590827+0000","last_peered":"2023-07-06T21:42:11.590827+0000","last_clean":"2023-07-06T21:42:11.590827+0000","last_became_active":"2023-07-06T21:42:10.603787+0000","last_became_peered":"2023-07-06T21:42:10.603787+0000","last_unstale":"2023-07-06T21:42:11.590827+0000","last_undegraded":"2023-07-06T21:42:11.590827+0000","last_fullsized":"2023-07-06T21:42:11.590827+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:58:16.849785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.608841+0000","last_change":"2023-07-06T21:42:10.606029+0000","last_active":"2023-07-06T21:42:10.608841+0000","last_peered":"2023-07-06T21:42:10.608841+0000","last_clean":"2023-07-06T21:42:10.608841+0000","last_became_active":"2023-07-06T21:42:10.604342+0000","last_became_peered":"2023-07-06T21:42:10.604342+0000","last_unstale":"2023-07-06T21:42:10.608841+0000","last_undegraded":"2023-07-06T21:42:10.608841+0000","last_fullsized":"2023-07-06T21:42:10.608841+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:40:27.493052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591675+0000","last_change":"2023-07-06T21:42:10.605090+0000","last_active":"2023-07-06T21:42:11.591675+0000","last_peered":"2023-07-06T21:42:11.591675+0000","last_clean":"2023-07-06T21:42:11.591675+0000","last_became_active":"2023-07-06T21:42:10.604757+0000","last_became_peered":"2023-07-06T21:42:10.604757+0000","last_unstale":"2023-07-06T21:42:11.591675+0000","last_undegraded":"2023-07-06T21:42:11.591675+0000","last_fullsized":"2023-07-06T21:42:11.591675+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:55.967710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.611194+0000","last_change":"2023-07-06T21:42:10.605227+0000","last_active":"2023-07-06T21:42:10.611194+0000","last_peered":"2023-07-06T21:42:10.611194+0000","last_clean":"2023-07-06T21:42:10.611194+0000","last_became_active":"2023-07-06T21:42:10.604932+0000","last_became_peered":"2023-07-06T21:42:10.604932+0000","last_unstale":"2023-07-06T21:42:10.611194+0000","last_undegraded":"2023-07-06T21:42:10.611194+0000","last_fullsized":"2023-07-06T21:42:10.611194+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:32.056656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.603801+0000","last_change":"2023-07-06T21:42:10.595947+0000","last_active":"2023-07-06T21:42:10.603801+0000","last_peered":"2023-07-06T21:42:10.603801+0000","last_clean":"2023-07-06T21:42:10.603801+0000","last_became_active":"2023-07-06T21:42:10.595698+0000","last_became_peered":"2023-07-06T21:42:10.595698+0000","last_unstale":"2023-07-06T21:42:10.603801+0000","last_undegraded":"2023-07-06T21:42:10.603801+0000","last_fullsized":"2023-07-06T21:42:10.603801+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:03:10.800847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590917+0000","last_change":"2023-07-06T21:42:10.606844+0000","last_active":"2023-07-06T21:42:11.590917+0000","last_peered":"2023-07-06T21:42:11.590917+0000","last_clean":"2023-07-06T21:42:11.590917+0000","last_became_active":"2023-07-06T21:42:10.606471+0000","last_became_peered":"2023-07-06T21:42:10.606471+0000","last_unstale":"2023-07-06T21:42:11.590917+0000","last_undegraded":"2023-07-06T21:42:11.590917+0000","last_fullsized":"2023-07-06T21:42:11.590917+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:52:48.938598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606204+0000","last_change":"2023-07-06T21:42:10.602650+0000","last_active":"2023-07-06T21:42:10.606204+0000","last_peered":"2023-07-06T21:42:10.606204+0000","last_clean":"2023-07-06T21:42:10.606204+0000","last_became_active":"2023-07-06T21:42:10.602364+0000","last_became_peered":"2023-07-06T21:42:10.602364+0000","last_unstale":"2023-07-06T21:42:10.606204+0000","last_undegraded":"2023-07-06T21:42:10.606204+0000","last_fullsized":"2023-07-06T21:42:10.606204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:14:26.542391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591081+0000","last_change":"2023-07-06T21:42:10.598716+0000","last_active":"2023-07-06T21:42:11.591081+0000","last_peered":"2023-07-06T21:42:11.591081+0000","last_clean":"2023-07-06T21:42:11.591081+0000","last_became_active":"2023-07-06T21:42:10.598418+0000","last_became_peered":"2023-07-06T21:42:10.598418+0000","last_unstale":"2023-07-06T21:42:11.591081+0000","last_undegraded":"2023-07-06T21:42:11.591081+0000","last_fullsized":"2023-07-06T21:42:11.591081+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:10.542591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590892+0000","last_change":"2023-07-06T21:42:10.598485+0000","last_active":"2023-07-06T21:42:11.590892+0000","last_peered":"2023-07-06T21:42:11.590892+0000","last_clean":"2023-07-06T21:42:11.590892+0000","last_became_active":"2023-07-06T21:42:10.598173+0000","last_became_peered":"2023-07-06T21:42:10.598173+0000","last_unstale":"2023-07-06T21:42:11.590892+0000","last_undegraded":"2023-07-06T21:42:11.590892+0000","last_fullsized":"2023-07-06T21:42:11.590892+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:30.099898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612496+0000","last_change":"2023-07-06T21:42:10.604690+0000","last_active":"2023-07-06T21:42:10.612496+0000","last_peered":"2023-07-06T21:42:10.612496+0000","last_clean":"2023-07-06T21:42:10.612496+0000","last_became_active":"2023-07-06T21:42:10.603767+0000","last_became_peered":"2023-07-06T21:42:10.603767+0000","last_unstale":"2023-07-06T21:42:10.612496+0000","last_undegraded":"2023-07-06T21:42:10.612496+0000","last_fullsized":"2023-07-06T21:42:10.612496+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:41:59.331696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.608784+0000","last_change":"2023-07-06T21:42:10.604339+0000","last_active":"2023-07-06T21:42:10.608784+0000","last_peered":"2023-07-06T21:42:10.608784+0000","last_clean":"2023-07-06T21:42:10.608784+0000","last_became_active":"2023-07-06T21:42:10.603872+0000","last_became_peered":"2023-07-06T21:42:10.603872+0000","last_unstale":"2023-07-06T21:42:10.608784+0000","last_undegraded":"2023-07-06T21:42:10.608784+0000","last_fullsized":"2023-07-06T21:42:10.608784+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:29:55.967710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619625+0000","last_change":"2023-07-06T21:42:10.613543+0000","last_active":"2023-07-06T21:42:10.619625+0000","last_peered":"2023-07-06T21:42:10.619625+0000","last_clean":"2023-07-06T21:42:10.619625+0000","last_became_active":"2023-07-06T21:42:10.613166+0000","last_became_peered":"2023-07-06T21:42:10.613166+0000","last_unstale":"2023-07-06T21:42:10.619625+0000","last_undegraded":"2023-07-06T21:42:10.619625+0000","last_fullsized":"2023-07-06T21:42:10.619625+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:27:00.253582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589066+0000","last_change":"2023-07-06T21:42:10.595764+0000","last_active":"2023-07-06T21:42:11.589066+0000","last_peered":"2023-07-06T21:42:11.589066+0000","last_clean":"2023-07-06T21:42:11.589066+0000","last_became_active":"2023-07-06T21:42:10.595468+0000","last_became_peered":"2023-07-06T21:42:10.595468+0000","last_unstale":"2023-07-06T21:42:11.589066+0000","last_undegraded":"2023-07-06T21:42:11.589066+0000","last_fullsized":"2023-07-06T21:42:11.589066+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:40:43.998094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609274+0000","last_change":"2023-07-06T21:42:10.607771+0000","last_active":"2023-07-06T21:42:10.609274+0000","last_peered":"2023-07-06T21:42:10.609274+0000","last_clean":"2023-07-06T21:42:10.609274+0000","last_became_active":"2023-07-06T21:42:10.607449+0000","last_became_peered":"2023-07-06T21:42:10.607449+0000","last_unstale":"2023-07-06T21:42:10.609274+0000","last_undegraded":"2023-07-06T21:42:10.609274+0000","last_fullsized":"2023-07-06T21:42:10.609274+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:38:06.944141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.620475+0000","last_change":"2023-07-06T21:42:10.616730+0000","last_active":"2023-07-06T21:42:10.620475+0000","last_peered":"2023-07-06T21:42:10.620475+0000","last_clean":"2023-07-06T21:42:10.620475+0000","last_became_active":"2023-07-06T21:42:10.616383+0000","last_became_peered":"2023-07-06T21:42:10.616383+0000","last_unstale":"2023-07-06T21:42:10.620475+0000","last_undegraded":"2023-07-06T21:42:10.620475+0000","last_fullsized":"2023-07-06T21:42:10.620475+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:30:44.679449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606375+0000","last_change":"2023-07-06T21:42:10.598413+0000","last_active":"2023-07-06T21:42:10.606375+0000","last_peered":"2023-07-06T21:42:10.606375+0000","last_clean":"2023-07-06T21:42:10.606375+0000","last_became_active":"2023-07-06T21:42:10.597804+0000","last_became_peered":"2023-07-06T21:42:10.597804+0000","last_unstale":"2023-07-06T21:42:10.606375+0000","last_undegraded":"2023-07-06T21:42:10.606375+0000","last_fullsized":"2023-07-06T21:42:10.606375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:25.654488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.615501+0000","last_change":"2023-07-06T21:42:10.610209+0000","last_active":"2023-07-06T21:42:10.615501+0000","last_peered":"2023-07-06T21:42:10.615501+0000","last_clean":"2023-07-06T21:42:10.615501+0000","last_became_active":"2023-07-06T21:42:10.609884+0000","last_became_peered":"2023-07-06T21:42:10.609884+0000","last_unstale":"2023-07-06T21:42:10.615501+0000","last_undegraded":"2023-07-06T21:42:10.615501+0000","last_fullsized":"2023-07-06T21:42:10.615501+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:56:24.004393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612366+0000","last_change":"2023-07-06T21:42:10.607800+0000","last_active":"2023-07-06T21:42:10.612366+0000","last_peered":"2023-07-06T21:42:10.612366+0000","last_clean":"2023-07-06T21:42:10.612366+0000","last_became_active":"2023-07-06T21:42:10.607299+0000","last_became_peered":"2023-07-06T21:42:10.607299+0000","last_unstale":"2023-07-06T21:42:10.612366+0000","last_undegraded":"2023-07-06T21:42:10.612366+0000","last_fullsized":"2023-07-06T21:42:10.612366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:11:11.235771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589313+0000","last_change":"2023-07-06T21:42:10.594998+0000","last_active":"2023-07-06T21:42:11.589313+0000","last_peered":"2023-07-06T21:42:11.589313+0000","last_clean":"2023-07-06T21:42:11.589313+0000","last_became_active":"2023-07-06T21:42:10.594743+0000","last_became_peered":"2023-07-06T21:42:10.594743+0000","last_unstale":"2023-07-06T21:42:11.589313+0000","last_undegraded":"2023-07-06T21:42:11.589313+0000","last_fullsized":"2023-07-06T21:42:11.589313+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:51:24.540760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591427+0000","last_change":"2023-07-06T21:42:10.608043+0000","last_active":"2023-07-06T21:42:11.591427+0000","last_peered":"2023-07-06T21:42:11.591427+0000","last_clean":"2023-07-06T21:42:11.591427+0000","last_became_active":"2023-07-06T21:42:10.607743+0000","last_became_peered":"2023-07-06T21:42:10.607743+0000","last_unstale":"2023-07-06T21:42:11.591427+0000","last_undegraded":"2023-07-06T21:42:11.591427+0000","last_fullsized":"2023-07-06T21:42:11.591427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:56:28.649508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612463+0000","last_change":"2023-07-06T21:42:10.608269+0000","last_active":"2023-07-06T21:42:10.612463+0000","last_peered":"2023-07-06T21:42:10.612463+0000","last_clean":"2023-07-06T21:42:10.612463+0000","last_became_active":"2023-07-06T21:42:10.608013+0000","last_became_peered":"2023-07-06T21:42:10.608013+0000","last_unstale":"2023-07-06T21:42:10.612463+0000","last_undegraded":"2023-07-06T21:42:10.612463+0000","last_fullsized":"2023-07-06T21:42:10.612463+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:03:10.800847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.589114+0000","last_change":"2023-07-06T21:42:10.605574+0000","last_active":"2023-07-06T21:42:11.589114+0000","last_peered":"2023-07-06T21:42:11.589114+0000","last_clean":"2023-07-06T21:42:11.589114+0000","last_became_active":"2023-07-06T21:42:10.605287+0000","last_became_peered":"2023-07-06T21:42:10.605287+0000","last_unstale":"2023-07-06T21:42:11.589114+0000","last_undegraded":"2023-07-06T21:42:11.589114+0000","last_fullsized":"2023-07-06T21:42:11.589114+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:01.158758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612162+0000","last_change":"2023-07-06T21:42:10.610723+0000","last_active":"2023-07-06T21:42:10.612162+0000","last_peered":"2023-07-06T21:42:10.612162+0000","last_clean":"2023-07-06T21:42:10.612162+0000","last_became_active":"2023-07-06T21:42:10.609369+0000","last_became_peered":"2023-07-06T21:42:10.609369+0000","last_unstale":"2023-07-06T21:42:10.612162+0000","last_undegraded":"2023-07-06T21:42:10.612162+0000","last_fullsized":"2023-07-06T21:42:10.612162+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:14.024714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619487+0000","last_change":"2023-07-06T21:42:10.610652+0000","last_active":"2023-07-06T21:42:10.619487+0000","last_peered":"2023-07-06T21:42:10.619487+0000","last_clean":"2023-07-06T21:42:10.619487+0000","last_became_active":"2023-07-06T21:42:10.610322+0000","last_became_peered":"2023-07-06T21:42:10.610322+0000","last_unstale":"2023-07-06T21:42:10.619487+0000","last_undegraded":"2023-07-06T21:42:10.619487+0000","last_fullsized":"2023-07-06T21:42:10.619487+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:12:01.721085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.588918+0000","last_change":"2023-07-06T21:42:10.593788+0000","last_active":"2023-07-06T21:42:11.588918+0000","last_peered":"2023-07-06T21:42:11.588918+0000","last_clean":"2023-07-06T21:42:11.588918+0000","last_became_active":"2023-07-06T21:42:10.593514+0000","last_became_peered":"2023-07-06T21:42:10.593514+0000","last_unstale":"2023-07-06T21:42:11.588918+0000","last_undegraded":"2023-07-06T21:42:11.588918+0000","last_fullsized":"2023-07-06T21:42:11.588918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:57:29.124985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.604539+0000","last_change":"2023-07-06T21:42:10.599454+0000","last_active":"2023-07-06T21:42:10.604539+0000","last_peered":"2023-07-06T21:42:10.604539+0000","last_clean":"2023-07-06T21:42:10.604539+0000","last_became_active":"2023-07-06T21:42:10.599126+0000","last_became_peered":"2023-07-06T21:42:10.599126+0000","last_unstale":"2023-07-06T21:42:10.604539+0000","last_undegraded":"2023-07-06T21:42:10.604539+0000","last_fullsized":"2023-07-06T21:42:10.604539+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:59:27.262227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612512+0000","last_change":"2023-07-06T21:42:10.609066+0000","last_active":"2023-07-06T21:42:10.612512+0000","last_peered":"2023-07-06T21:42:10.612512+0000","last_clean":"2023-07-06T21:42:10.612512+0000","last_became_active":"2023-07-06T21:42:10.608817+0000","last_became_peered":"2023-07-06T21:42:10.608817+0000","last_unstale":"2023-07-06T21:42:10.612512+0000","last_undegraded":"2023-07-06T21:42:10.612512+0000","last_fullsized":"2023-07-06T21:42:10.612512+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:56.395390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"52'4","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.614153+0000","last_change":"2023-07-06T21:42:10.106103+0000","last_active":"2023-07-06T21:42:10.614153+0000","last_peered":"2023-07-06T21:42:10.614153+0000","last_clean":"2023-07-06T21:42:10.614153+0000","last_became_active":"2023-07-06T21:42:10.105766+0000","last_became_peered":"2023-07-06T21:42:10.105766+0000","last_unstale":"2023-07-06T21:42:10.614153+0000","last_undegraded":"2023-07-06T21:42:10.614153+0000","last_fullsized":"2023-07-06T21:42:10.614153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:54:50.441575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612832+0000","last_change":"2023-07-06T21:42:10.611656+0000","last_active":"2023-07-06T21:42:10.612832+0000","last_peered":"2023-07-06T21:42:10.612832+0000","last_clean":"2023-07-06T21:42:10.612832+0000","last_became_active":"2023-07-06T21:42:10.611092+0000","last_became_peered":"2023-07-06T21:42:10.611092+0000","last_unstale":"2023-07-06T21:42:10.612832+0000","last_undegraded":"2023-07-06T21:42:10.612832+0000","last_fullsized":"2023-07-06T21:42:10.612832+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:37:33.183332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.613941+0000","last_change":"2023-07-06T21:42:10.109333+0000","last_active":"2023-07-06T21:42:10.613941+0000","last_peered":"2023-07-06T21:42:10.613941+0000","last_clean":"2023-07-06T21:42:10.613941+0000","last_became_active":"2023-07-06T21:42:10.109030+0000","last_became_peered":"2023-07-06T21:42:10.109030+0000","last_unstale":"2023-07-06T21:42:10.613941+0000","last_undegraded":"2023-07-06T21:42:10.613941+0000","last_fullsized":"2023-07-06T21:42:10.613941+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:04:57.920104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612592+0000","last_change":"2023-07-06T21:42:10.603158+0000","last_active":"2023-07-06T21:42:10.612592+0000","last_peered":"2023-07-06T21:42:10.612592+0000","last_clean":"2023-07-06T21:42:10.612592+0000","last_became_active":"2023-07-06T21:42:10.602804+0000","last_became_peered":"2023-07-06T21:42:10.602804+0000","last_unstale":"2023-07-06T21:42:10.612592+0000","last_undegraded":"2023-07-06T21:42:10.612592+0000","last_fullsized":"2023-07-06T21:42:10.612592+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:47:04.939652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612172+0000","last_change":"2023-07-06T21:42:10.106491+0000","last_active":"2023-07-06T21:42:10.612172+0000","last_peered":"2023-07-06T21:42:10.612172+0000","last_clean":"2023-07-06T21:42:10.612172+0000","last_became_active":"2023-07-06T21:42:10.106214+0000","last_became_peered":"2023-07-06T21:42:10.106214+0000","last_unstale":"2023-07-06T21:42:10.612172+0000","last_undegraded":"2023-07-06T21:42:10.612172+0000","last_fullsized":"2023-07-06T21:42:10.612172+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:31.046674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612342+0000","last_change":"2023-07-06T21:42:10.605528+0000","last_active":"2023-07-06T21:42:10.612342+0000","last_peered":"2023-07-06T21:42:10.612342+0000","last_clean":"2023-07-06T21:42:10.612342+0000","last_became_active":"2023-07-06T21:42:10.605060+0000","last_became_peered":"2023-07-06T21:42:10.605060+0000","last_unstale":"2023-07-06T21:42:10.612342+0000","last_undegraded":"2023-07-06T21:42:10.612342+0000","last_fullsized":"2023-07-06T21:42:10.612342+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:37:33.183332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"52'2","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591410+0000","last_change":"2023-07-06T21:42:10.105700+0000","last_active":"2023-07-06T21:42:11.591410+0000","last_peered":"2023-07-06T21:42:11.591410+0000","last_clean":"2023-07-06T21:42:11.591410+0000","last_became_active":"2023-07-06T21:42:10.105401+0000","last_became_peered":"2023-07-06T21:42:10.105401+0000","last_unstale":"2023-07-06T21:42:11.591410+0000","last_undegraded":"2023-07-06T21:42:11.591410+0000","last_fullsized":"2023-07-06T21:42:11.591410+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:30:09.460291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606355+0000","last_change":"2023-07-06T21:42:10.598231+0000","last_active":"2023-07-06T21:42:10.606355+0000","last_peered":"2023-07-06T21:42:10.606355+0000","last_clean":"2023-07-06T21:42:10.606355+0000","last_became_active":"2023-07-06T21:42:10.597447+0000","last_became_peered":"2023-07-06T21:42:10.597447+0000","last_unstale":"2023-07-06T21:42:10.606355+0000","last_undegraded":"2023-07-06T21:42:10.606355+0000","last_fullsized":"2023-07-06T21:42:10.606355+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:12:57.509331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"52'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591408+0000","last_change":"2023-07-06T21:42:10.099044+0000","last_active":"2023-07-06T21:42:11.591408+0000","last_peered":"2023-07-06T21:42:11.591408+0000","last_clean":"2023-07-06T21:42:11.591408+0000","last_became_active":"2023-07-06T21:42:10.098642+0000","last_became_peered":"2023-07-06T21:42:10.098642+0000","last_unstale":"2023-07-06T21:42:11.591408+0000","last_undegraded":"2023-07-06T21:42:11.591408+0000","last_fullsized":"2023-07-06T21:42:11.591408+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:35.393145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610813+0000","last_change":"2023-07-06T21:42:10.603023+0000","last_active":"2023-07-06T21:42:10.610813+0000","last_peered":"2023-07-06T21:42:10.610813+0000","last_clean":"2023-07-06T21:42:10.610813+0000","last_became_active":"2023-07-06T21:42:10.602775+0000","last_became_peered":"2023-07-06T21:42:10.602775+0000","last_unstale":"2023-07-06T21:42:10.610813+0000","last_undegraded":"2023-07-06T21:42:10.610813+0000","last_fullsized":"2023-07-06T21:42:10.610813+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:40:07.262353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.607485+0000","last_change":"2023-07-06T21:42:10.104771+0000","last_active":"2023-07-06T21:42:10.607485+0000","last_peered":"2023-07-06T21:42:10.607485+0000","last_clean":"2023-07-06T21:42:10.607485+0000","last_became_active":"2023-07-06T21:42:10.104426+0000","last_became_peered":"2023-07-06T21:42:10.104426+0000","last_unstale":"2023-07-06T21:42:10.607485+0000","last_undegraded":"2023-07-06T21:42:10.607485+0000","last_fullsized":"2023-07-06T21:42:10.607485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:50.666692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.617406+0000","last_change":"2023-07-06T21:42:10.613088+0000","last_active":"2023-07-06T21:42:10.617406+0000","last_peered":"2023-07-06T21:42:10.617406+0000","last_clean":"2023-07-06T21:42:10.617406+0000","last_became_active":"2023-07-06T21:42:10.612791+0000","last_became_peered":"2023-07-06T21:42:10.612791+0000","last_unstale":"2023-07-06T21:42:10.617406+0000","last_undegraded":"2023-07-06T21:42:10.617406+0000","last_fullsized":"2023-07-06T21:42:10.617406+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:10:36.332321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590815+0000","last_change":"2023-07-06T21:42:10.100741+0000","last_active":"2023-07-06T21:42:11.590815+0000","last_peered":"2023-07-06T21:42:11.590815+0000","last_clean":"2023-07-06T21:42:11.590815+0000","last_became_active":"2023-07-06T21:42:10.100453+0000","last_became_peered":"2023-07-06T21:42:10.100453+0000","last_unstale":"2023-07-06T21:42:11.590815+0000","last_undegraded":"2023-07-06T21:42:11.590815+0000","last_fullsized":"2023-07-06T21:42:11.590815+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:33:22.640052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609719+0000","last_change":"2023-07-06T21:42:10.608496+0000","last_active":"2023-07-06T21:42:10.609719+0000","last_peered":"2023-07-06T21:42:10.609719+0000","last_clean":"2023-07-06T21:42:10.609719+0000","last_became_active":"2023-07-06T21:42:10.608027+0000","last_became_peered":"2023-07-06T21:42:10.608027+0000","last_unstale":"2023-07-06T21:42:10.609719+0000","last_undegraded":"2023-07-06T21:42:10.609719+0000","last_fullsized":"2023-07-06T21:42:10.609719+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:27:24.653218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590654+0000","last_change":"2023-07-06T21:42:10.106387+0000","last_active":"2023-07-06T21:42:11.590654+0000","last_peered":"2023-07-06T21:42:11.590654+0000","last_clean":"2023-07-06T21:42:11.590654+0000","last_became_active":"2023-07-06T21:42:10.106024+0000","last_became_peered":"2023-07-06T21:42:10.106024+0000","last_unstale":"2023-07-06T21:42:11.590654+0000","last_undegraded":"2023-07-06T21:42:11.590654+0000","last_fullsized":"2023-07-06T21:42:11.590654+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:03.166828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591188+0000","last_change":"2023-07-06T21:42:10.598376+0000","last_active":"2023-07-06T21:42:11.591188+0000","last_peered":"2023-07-06T21:42:11.591188+0000","last_clean":"2023-07-06T21:42:11.591188+0000","last_became_active":"2023-07-06T21:42:10.598075+0000","last_became_peered":"2023-07-06T21:42:10.598075+0000","last_unstale":"2023-07-06T21:42:11.591188+0000","last_undegraded":"2023-07-06T21:42:11.591188+0000","last_fullsized":"2023-07-06T21:42:11.591188+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:25.572938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.599852+0000","last_change":"2023-07-06T21:42:10.107114+0000","last_active":"2023-07-06T21:42:10.599852+0000","last_peered":"2023-07-06T21:42:10.599852+0000","last_clean":"2023-07-06T21:42:10.599852+0000","last_became_active":"2023-07-06T21:42:10.106823+0000","last_became_peered":"2023-07-06T21:42:10.106823+0000","last_unstale":"2023-07-06T21:42:10.599852+0000","last_undegraded":"2023-07-06T21:42:10.599852+0000","last_fullsized":"2023-07-06T21:42:10.599852+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:54:40.407128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619600+0000","last_change":"2023-07-06T21:42:10.612583+0000","last_active":"2023-07-06T21:42:10.619600+0000","last_peered":"2023-07-06T21:42:10.619600+0000","last_clean":"2023-07-06T21:42:10.619600+0000","last_became_active":"2023-07-06T21:42:10.612182+0000","last_became_peered":"2023-07-06T21:42:10.612182+0000","last_unstale":"2023-07-06T21:42:10.619600+0000","last_undegraded":"2023-07-06T21:42:10.619600+0000","last_fullsized":"2023-07-06T21:42:10.619600+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:51:48.928642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.586646+0000","last_change":"2023-07-06T21:42:10.103885+0000","last_active":"2023-07-06T21:42:10.586646+0000","last_peered":"2023-07-06T21:42:10.586646+0000","last_clean":"2023-07-06T21:42:10.586646+0000","last_became_active":"2023-07-06T21:42:10.103485+0000","last_became_peered":"2023-07-06T21:42:10.103485+0000","last_unstale":"2023-07-06T21:42:10.586646+0000","last_undegraded":"2023-07-06T21:42:10.586646+0000","last_fullsized":"2023-07-06T21:42:10.586646+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:58:15.839803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610317+0000","last_change":"2023-07-06T21:42:10.601953+0000","last_active":"2023-07-06T21:42:10.610317+0000","last_peered":"2023-07-06T21:42:10.610317+0000","last_clean":"2023-07-06T21:42:10.610317+0000","last_became_active":"2023-07-06T21:42:10.601683+0000","last_became_peered":"2023-07-06T21:42:10.601683+0000","last_unstale":"2023-07-06T21:42:10.610317+0000","last_undegraded":"2023-07-06T21:42:10.610317+0000","last_fullsized":"2023-07-06T21:42:10.610317+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:14.358591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"52'3","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.614457+0000","last_change":"2023-07-06T21:42:10.111931+0000","last_active":"2023-07-06T21:42:10.614457+0000","last_peered":"2023-07-06T21:42:10.614457+0000","last_clean":"2023-07-06T21:42:10.614457+0000","last_became_active":"2023-07-06T21:42:10.111673+0000","last_became_peered":"2023-07-06T21:42:10.111673+0000","last_unstale":"2023-07-06T21:42:10.614457+0000","last_undegraded":"2023-07-06T21:42:10.614457+0000","last_fullsized":"2023-07-06T21:42:10.614457+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:33:16.351712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619691+0000","last_change":"2023-07-06T21:42:10.614433+0000","last_active":"2023-07-06T21:42:10.619691+0000","last_peered":"2023-07-06T21:42:10.619691+0000","last_clean":"2023-07-06T21:42:10.619691+0000","last_became_active":"2023-07-06T21:42:10.613807+0000","last_became_peered":"2023-07-06T21:42:10.613807+0000","last_unstale":"2023-07-06T21:42:10.619691+0000","last_undegraded":"2023-07-06T21:42:10.619691+0000","last_fullsized":"2023-07-06T21:42:10.619691+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:27:12.933707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"52'6","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591689+0000","last_change":"2023-07-06T21:42:10.108114+0000","last_active":"2023-07-06T21:42:11.591689+0000","last_peered":"2023-07-06T21:42:11.591689+0000","last_clean":"2023-07-06T21:42:11.591689+0000","last_became_active":"2023-07-06T21:42:10.107799+0000","last_became_peered":"2023-07-06T21:42:10.107799+0000","last_unstale":"2023-07-06T21:42:11.591689+0000","last_undegraded":"2023-07-06T21:42:11.591689+0000","last_fullsized":"2023-07-06T21:42:11.591689+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:38:05.934159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590960+0000","last_change":"2023-07-06T21:42:10.596784+0000","last_active":"2023-07-06T21:42:11.590960+0000","last_peered":"2023-07-06T21:42:11.590960+0000","last_clean":"2023-07-06T21:42:11.590960+0000","last_became_active":"2023-07-06T21:42:10.596479+0000","last_became_peered":"2023-07-06T21:42:10.596479+0000","last_unstale":"2023-07-06T21:42:11.590960+0000","last_undegraded":"2023-07-06T21:42:11.590960+0000","last_fullsized":"2023-07-06T21:42:11.590960+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:48:42.014991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609072+0000","last_change":"2023-07-06T21:42:10.103371+0000","last_active":"2023-07-06T21:42:10.609072+0000","last_peered":"2023-07-06T21:42:10.609072+0000","last_clean":"2023-07-06T21:42:10.609072+0000","last_became_active":"2023-07-06T21:42:10.102943+0000","last_became_peered":"2023-07-06T21:42:10.102943+0000","last_unstale":"2023-07-06T21:42:10.609072+0000","last_undegraded":"2023-07-06T21:42:10.609072+0000","last_fullsized":"2023-07-06T21:42:10.609072+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:29.089916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619483+0000","last_change":"2023-07-06T21:42:10.611021+0000","last_active":"2023-07-06T21:42:10.619483+0000","last_peered":"2023-07-06T21:42:10.619483+0000","last_clean":"2023-07-06T21:42:10.619483+0000","last_became_active":"2023-07-06T21:42:10.610749+0000","last_became_peered":"2023-07-06T21:42:10.610749+0000","last_unstale":"2023-07-06T21:42:10.619483+0000","last_undegraded":"2023-07-06T21:42:10.619483+0000","last_fullsized":"2023-07-06T21:42:10.619483+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:00:27.140587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590435+0000","last_change":"2023-07-06T21:42:10.108145+0000","last_active":"2023-07-06T21:42:10.590435+0000","last_peered":"2023-07-06T21:42:10.590435+0000","last_clean":"2023-07-06T21:42:10.590435+0000","last_became_active":"2023-07-06T21:42:10.107833+0000","last_became_peered":"2023-07-06T21:42:10.107833+0000","last_unstale":"2023-07-06T21:42:10.590435+0000","last_undegraded":"2023-07-06T21:42:10.590435+0000","last_fullsized":"2023-07-06T21:42:10.590435+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:38:33.511306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590510+0000","last_change":"2023-07-06T21:42:10.605169+0000","last_active":"2023-07-06T21:42:11.590510+0000","last_peered":"2023-07-06T21:42:11.590510+0000","last_clean":"2023-07-06T21:42:11.590510+0000","last_became_active":"2023-07-06T21:42:10.604866+0000","last_became_peered":"2023-07-06T21:42:10.604866+0000","last_unstale":"2023-07-06T21:42:11.590510+0000","last_undegraded":"2023-07-06T21:42:11.590510+0000","last_fullsized":"2023-07-06T21:42:11.590510+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:51:14.024714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590957+0000","last_change":"2023-07-06T21:42:10.103020+0000","last_active":"2023-07-06T21:42:11.590957+0000","last_peered":"2023-07-06T21:42:11.590957+0000","last_clean":"2023-07-06T21:42:11.590957+0000","last_became_active":"2023-07-06T21:42:10.102679+0000","last_became_peered":"2023-07-06T21:42:10.102679+0000","last_unstale":"2023-07-06T21:42:11.590957+0000","last_undegraded":"2023-07-06T21:42:11.590957+0000","last_fullsized":"2023-07-06T21:42:11.590957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:40:29.483560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619466+0000","last_change":"2023-07-06T21:42:10.610661+0000","last_active":"2023-07-06T21:42:10.619466+0000","last_peered":"2023-07-06T21:42:10.619466+0000","last_clean":"2023-07-06T21:42:10.619466+0000","last_became_active":"2023-07-06T21:42:10.609061+0000","last_became_peered":"2023-07-06T21:42:10.609061+0000","last_unstale":"2023-07-06T21:42:10.619466+0000","last_undegraded":"2023-07-06T21:42:10.619466+0000","last_fullsized":"2023-07-06T21:42:10.619466+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:28:12.543043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"52'8","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.616638+0000","last_change":"2023-07-06T21:42:10.109338+0000","last_active":"2023-07-06T21:42:10.616638+0000","last_peered":"2023-07-06T21:42:10.616638+0000","last_clean":"2023-07-06T21:42:10.616638+0000","last_became_active":"2023-07-06T21:42:10.109020+0000","last_became_peered":"2023-07-06T21:42:10.109020+0000","last_unstale":"2023-07-06T21:42:10.616638+0000","last_undegraded":"2023-07-06T21:42:10.616638+0000","last_fullsized":"2023-07-06T21:42:10.616638+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:34:17.346674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591597+0000","last_change":"2023-07-06T21:42:10.600611+0000","last_active":"2023-07-06T21:42:11.591597+0000","last_peered":"2023-07-06T21:42:11.591597+0000","last_clean":"2023-07-06T21:42:11.591597+0000","last_became_active":"2023-07-06T21:42:10.600106+0000","last_became_peered":"2023-07-06T21:42:10.600106+0000","last_unstale":"2023-07-06T21:42:11.591597+0000","last_undegraded":"2023-07-06T21:42:11.591597+0000","last_fullsized":"2023-07-06T21:42:11.591597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:22:18.746595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610189+0000","last_change":"2023-07-06T21:42:10.109503+0000","last_active":"2023-07-06T21:42:10.610189+0000","last_peered":"2023-07-06T21:42:10.610189+0000","last_clean":"2023-07-06T21:42:10.610189+0000","last_became_active":"2023-07-06T21:42:10.109208+0000","last_became_peered":"2023-07-06T21:42:10.109208+0000","last_unstale":"2023-07-06T21:42:10.610189+0000","last_undegraded":"2023-07-06T21:42:10.610189+0000","last_fullsized":"2023-07-06T21:42:10.610189+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:05:07.616090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591659+0000","last_change":"2023-07-06T21:42:10.615022+0000","last_active":"2023-07-06T21:42:11.591659+0000","last_peered":"2023-07-06T21:42:11.591659+0000","last_clean":"2023-07-06T21:42:11.591659+0000","last_became_active":"2023-07-06T21:42:10.614673+0000","last_became_peered":"2023-07-06T21:42:10.614673+0000","last_unstale":"2023-07-06T21:42:11.591659+0000","last_undegraded":"2023-07-06T21:42:11.591659+0000","last_fullsized":"2023-07-06T21:42:11.591659+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T08:47:42.190957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590153+0000","last_change":"2023-07-06T21:42:10.106050+0000","last_active":"2023-07-06T21:42:10.590153+0000","last_peered":"2023-07-06T21:42:10.590153+0000","last_clean":"2023-07-06T21:42:10.590153+0000","last_became_active":"2023-07-06T21:42:10.105796+0000","last_became_peered":"2023-07-06T21:42:10.105796+0000","last_unstale":"2023-07-06T21:42:10.590153+0000","last_undegraded":"2023-07-06T21:42:10.590153+0000","last_fullsized":"2023-07-06T21:42:10.590153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:38.363754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.611233+0000","last_change":"2023-07-06T21:42:10.604268+0000","last_active":"2023-07-06T21:42:10.611233+0000","last_peered":"2023-07-06T21:42:10.611233+0000","last_clean":"2023-07-06T21:42:10.611233+0000","last_became_active":"2023-07-06T21:42:10.603968+0000","last_became_peered":"2023-07-06T21:42:10.603968+0000","last_unstale":"2023-07-06T21:42:10.611233+0000","last_undegraded":"2023-07-06T21:42:10.611233+0000","last_fullsized":"2023-07-06T21:42:10.611233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:47:04.939652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590195+0000","last_change":"2023-07-06T21:42:10.101939+0000","last_active":"2023-07-06T21:42:10.590195+0000","last_peered":"2023-07-06T21:42:10.590195+0000","last_clean":"2023-07-06T21:42:10.590195+0000","last_became_active":"2023-07-06T21:42:10.101645+0000","last_became_peered":"2023-07-06T21:42:10.101645+0000","last_unstale":"2023-07-06T21:42:10.590195+0000","last_undegraded":"2023-07-06T21:42:10.590195+0000","last_fullsized":"2023-07-06T21:42:10.590195+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:16:48.900805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612322+0000","last_change":"2023-07-06T21:42:10.605303+0000","last_active":"2023-07-06T21:42:10.612322+0000","last_peered":"2023-07-06T21:42:10.612322+0000","last_clean":"2023-07-06T21:42:10.612322+0000","last_became_active":"2023-07-06T21:42:10.604761+0000","last_became_peered":"2023-07-06T21:42:10.604761+0000","last_unstale":"2023-07-06T21:42:10.612322+0000","last_undegraded":"2023-07-06T21:42:10.612322+0000","last_fullsized":"2023-07-06T21:42:10.612322+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T05:22:21.583971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591059+0000","last_change":"2023-07-06T21:42:10.100770+0000","last_active":"2023-07-06T21:42:11.591059+0000","last_peered":"2023-07-06T21:42:11.591059+0000","last_clean":"2023-07-06T21:42:11.591059+0000","last_became_active":"2023-07-06T21:42:10.100490+0000","last_became_peered":"2023-07-06T21:42:10.100490+0000","last_unstale":"2023-07-06T21:42:11.591059+0000","last_undegraded":"2023-07-06T21:42:11.591059+0000","last_fullsized":"2023-07-06T21:42:11.591059+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:33:01.646226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.596509+0000","last_change":"2023-07-06T21:42:10.594392+0000","last_active":"2023-07-06T21:42:10.596509+0000","last_peered":"2023-07-06T21:42:10.596509+0000","last_clean":"2023-07-06T21:42:10.596509+0000","last_became_active":"2023-07-06T21:42:10.594141+0000","last_became_peered":"2023-07-06T21:42:10.594141+0000","last_unstale":"2023-07-06T21:42:10.596509+0000","last_undegraded":"2023-07-06T21:42:10.596509+0000","last_fullsized":"2023-07-06T21:42:10.596509+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:59:04.176809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.588657+0000","last_change":"2023-07-06T21:42:10.109157+0000","last_active":"2023-07-06T21:42:10.588657+0000","last_peered":"2023-07-06T21:42:10.588657+0000","last_clean":"2023-07-06T21:42:10.588657+0000","last_became_active":"2023-07-06T21:42:10.108868+0000","last_became_peered":"2023-07-06T21:42:10.108868+0000","last_unstale":"2023-07-06T21:42:10.588657+0000","last_undegraded":"2023-07-06T21:42:10.588657+0000","last_fullsized":"2023-07-06T21:42:10.588657+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:08:47.457572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619543+0000","last_change":"2023-07-06T21:42:10.611408+0000","last_active":"2023-07-06T21:42:10.619543+0000","last_peered":"2023-07-06T21:42:10.619543+0000","last_clean":"2023-07-06T21:42:10.619543+0000","last_became_active":"2023-07-06T21:42:10.610951+0000","last_became_peered":"2023-07-06T21:42:10.610951+0000","last_unstale":"2023-07-06T21:42:10.619543+0000","last_undegraded":"2023-07-06T21:42:10.619543+0000","last_fullsized":"2023-07-06T21:42:10.619543+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:46:45.859395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591153+0000","last_change":"2023-07-06T21:42:10.106381+0000","last_active":"2023-07-06T21:42:11.591153+0000","last_peered":"2023-07-06T21:42:11.591153+0000","last_clean":"2023-07-06T21:42:11.591153+0000","last_became_active":"2023-07-06T21:42:10.106075+0000","last_became_peered":"2023-07-06T21:42:10.106075+0000","last_unstale":"2023-07-06T21:42:11.591153+0000","last_undegraded":"2023-07-06T21:42:11.591153+0000","last_fullsized":"2023-07-06T21:42:11.591153+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:12:56.499349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612038+0000","last_change":"2023-07-06T21:42:10.605265+0000","last_active":"2023-07-06T21:42:10.612038+0000","last_peered":"2023-07-06T21:42:10.612038+0000","last_clean":"2023-07-06T21:42:10.612038+0000","last_became_active":"2023-07-06T21:42:10.604930+0000","last_became_peered":"2023-07-06T21:42:10.604930+0000","last_unstale":"2023-07-06T21:42:10.612038+0000","last_undegraded":"2023-07-06T21:42:10.612038+0000","last_fullsized":"2023-07-06T21:42:10.612038+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:39.373736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.586609+0000","last_change":"2023-07-06T21:42:10.107339+0000","last_active":"2023-07-06T21:42:10.586609+0000","last_peered":"2023-07-06T21:42:10.586609+0000","last_clean":"2023-07-06T21:42:10.586609+0000","last_became_active":"2023-07-06T21:42:10.107061+0000","last_became_peered":"2023-07-06T21:42:10.107061+0000","last_unstale":"2023-07-06T21:42:10.586609+0000","last_undegraded":"2023-07-06T21:42:10.586609+0000","last_fullsized":"2023-07-06T21:42:10.586609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:11:55.385408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.609699+0000","last_change":"2023-07-06T21:42:10.608287+0000","last_active":"2023-07-06T21:42:10.609699+0000","last_peered":"2023-07-06T21:42:10.609699+0000","last_clean":"2023-07-06T21:42:10.609699+0000","last_became_active":"2023-07-06T21:42:10.607617+0000","last_became_peered":"2023-07-06T21:42:10.607617+0000","last_unstale":"2023-07-06T21:42:10.609699+0000","last_undegraded":"2023-07-06T21:42:10.609699+0000","last_fullsized":"2023-07-06T21:42:10.609699+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:49:05.964671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.592796+0000","last_change":"2023-07-06T21:42:10.098590+0000","last_active":"2023-07-06T21:42:10.592796+0000","last_peered":"2023-07-06T21:42:10.592796+0000","last_clean":"2023-07-06T21:42:10.592796+0000","last_became_active":"2023-07-06T21:42:10.098111+0000","last_became_peered":"2023-07-06T21:42:10.098111+0000","last_unstale":"2023-07-06T21:42:10.592796+0000","last_undegraded":"2023-07-06T21:42:10.592796+0000","last_fullsized":"2023-07-06T21:42:10.592796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:00:24.466871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591603+0000","last_change":"2023-07-06T21:42:10.601307+0000","last_active":"2023-07-06T21:42:11.591603+0000","last_peered":"2023-07-06T21:42:11.591603+0000","last_clean":"2023-07-06T21:42:11.591603+0000","last_became_active":"2023-07-06T21:42:10.600999+0000","last_became_peered":"2023-07-06T21:42:10.600999+0000","last_unstale":"2023-07-06T21:42:11.591603+0000","last_undegraded":"2023-07-06T21:42:11.591603+0000","last_fullsized":"2023-07-06T21:42:11.591603+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:13:18.541193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.607379+0000","last_change":"2023-07-06T21:42:10.106022+0000","last_active":"2023-07-06T21:42:10.607379+0000","last_peered":"2023-07-06T21:42:10.607379+0000","last_clean":"2023-07-06T21:42:10.607379+0000","last_became_active":"2023-07-06T21:42:10.105743+0000","last_became_peered":"2023-07-06T21:42:10.105743+0000","last_unstale":"2023-07-06T21:42:10.607379+0000","last_undegraded":"2023-07-06T21:42:10.607379+0000","last_fullsized":"2023-07-06T21:42:10.607379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:13.348609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.587414+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:42:10.587414+0000","last_peered":"2023-07-06T21:42:10.587414+0000","last_clean":"2023-07-06T21:42:10.587414+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:42:10.587414+0000","last_undegraded":"2023-07-06T21:42:10.587414+0000","last_fullsized":"2023-07-06T21:42:10.587414+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:15:26.222928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.612369+0000","last_change":"2023-07-06T21:42:10.607093+0000","last_active":"2023-07-06T21:42:10.612369+0000","last_peered":"2023-07-06T21:42:10.612369+0000","last_clean":"2023-07-06T21:42:10.612369+0000","last_became_active":"2023-07-06T21:42:10.603592+0000","last_became_peered":"2023-07-06T21:42:10.603592+0000","last_unstale":"2023-07-06T21:42:10.612369+0000","last_undegraded":"2023-07-06T21:42:10.612369+0000","last_fullsized":"2023-07-06T21:42:10.612369+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:04:09.902720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590787+0000","last_change":"2023-07-06T21:42:10.103313+0000","last_active":"2023-07-06T21:42:11.590787+0000","last_peered":"2023-07-06T21:42:11.590787+0000","last_clean":"2023-07-06T21:42:11.590787+0000","last_became_active":"2023-07-06T21:42:10.102646+0000","last_became_peered":"2023-07-06T21:42:10.102646+0000","last_unstale":"2023-07-06T21:42:11.590787+0000","last_undegraded":"2023-07-06T21:42:11.590787+0000","last_fullsized":"2023-07-06T21:42:11.590787+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:19:30.095342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591063+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:42:11.591063+0000","last_peered":"2023-07-06T21:42:11.591063+0000","last_clean":"2023-07-06T21:42:11.591063+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:42:11.591063+0000","last_undegraded":"2023-07-06T21:42:11.591063+0000","last_fullsized":"2023-07-06T21:42:11.591063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:07:35.603909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591278+0000","last_change":"2023-07-06T21:42:10.602481+0000","last_active":"2023-07-06T21:42:11.591278+0000","last_peered":"2023-07-06T21:42:11.591278+0000","last_clean":"2023-07-06T21:42:11.591278+0000","last_became_active":"2023-07-06T21:42:10.602127+0000","last_became_peered":"2023-07-06T21:42:10.602127+0000","last_unstale":"2023-07-06T21:42:11.591278+0000","last_undegraded":"2023-07-06T21:42:11.591278+0000","last_fullsized":"2023-07-06T21:42:11.591278+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:54:23.563056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"52'2","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.610545+0000","last_change":"2023-07-06T21:42:10.103206+0000","last_active":"2023-07-06T21:42:10.610545+0000","last_peered":"2023-07-06T21:42:10.610545+0000","last_clean":"2023-07-06T21:42:10.610545+0000","last_became_active":"2023-07-06T21:42:10.102654+0000","last_became_peered":"2023-07-06T21:42:10.102654+0000","last_unstale":"2023-07-06T21:42:10.610545+0000","last_undegraded":"2023-07-06T21:42:10.610545+0000","last_fullsized":"2023-07-06T21:42:10.610545+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:25:24.562956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":72,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.588854+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:42:11.588854+0000","last_peered":"2023-07-06T21:42:11.588854+0000","last_clean":"2023-07-06T21:42:11.588854+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:42:11.588854+0000","last_undegraded":"2023-07-06T21:42:11.588854+0000","last_fullsized":"2023-07-06T21:42:11.588854+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:41:08.531361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.608624+0000","last_change":"2023-07-06T21:42:10.602370+0000","last_active":"2023-07-06T21:42:10.608624+0000","last_peered":"2023-07-06T21:42:10.608624+0000","last_clean":"2023-07-06T21:42:10.608624+0000","last_became_active":"2023-07-06T21:42:10.602011+0000","last_became_peered":"2023-07-06T21:42:10.602011+0000","last_unstale":"2023-07-06T21:42:10.608624+0000","last_undegraded":"2023-07-06T21:42:10.608624+0000","last_fullsized":"2023-07-06T21:42:10.608624+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:38:36.403127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.595197+0000","last_change":"2023-07-06T21:42:10.106213+0000","last_active":"2023-07-06T21:42:10.595197+0000","last_peered":"2023-07-06T21:42:10.595197+0000","last_clean":"2023-07-06T21:42:10.595197+0000","last_became_active":"2023-07-06T21:42:10.105939+0000","last_became_peered":"2023-07-06T21:42:10.105939+0000","last_unstale":"2023-07-06T21:42:10.595197+0000","last_undegraded":"2023-07-06T21:42:10.595197+0000","last_fullsized":"2023-07-06T21:42:10.595197+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:10:17.223996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":78,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.590846+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:42:11.590846+0000","last_peered":"2023-07-06T21:42:11.590846+0000","last_clean":"2023-07-06T21:42:11.590846+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:42:11.590846+0000","last_undegraded":"2023-07-06T21:42:11.590846+0000","last_fullsized":"2023-07-06T21:42:11.590846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:49:13.828393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.598075+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:42:10.598075+0000","last_peered":"2023-07-06T21:42:10.598075+0000","last_clean":"2023-07-06T21:42:10.598075+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:42:10.598075+0000","last_undegraded":"2023-07-06T21:42:10.598075+0000","last_fullsized":"2023-07-06T21:42:10.598075+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:56:02.236521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.611714+0000","last_change":"2023-07-06T21:42:10.602778+0000","last_active":"2023-07-06T21:42:10.611714+0000","last_peered":"2023-07-06T21:42:10.611714+0000","last_clean":"2023-07-06T21:42:10.611714+0000","last_became_active":"2023-07-06T21:42:10.602495+0000","last_became_peered":"2023-07-06T21:42:10.602495+0000","last_unstale":"2023-07-06T21:42:10.611714+0000","last_undegraded":"2023-07-06T21:42:10.611714+0000","last_fullsized":"2023-07-06T21:42:10.611714+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:01:12.971613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.587950+0000","last_change":"2023-07-06T21:42:10.106963+0000","last_active":"2023-07-06T21:42:10.587950+0000","last_peered":"2023-07-06T21:42:10.587950+0000","last_clean":"2023-07-06T21:42:10.587950+0000","last_became_active":"2023-07-06T21:42:10.106586+0000","last_became_peered":"2023-07-06T21:42:10.106586+0000","last_unstale":"2023-07-06T21:42:10.587950+0000","last_undegraded":"2023-07-06T21:42:10.587950+0000","last_fullsized":"2023-07-06T21:42:10.587950+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:33:50.666692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.597945+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:42:10.597945+0000","last_peered":"2023-07-06T21:42:10.597945+0000","last_clean":"2023-07-06T21:42:10.597945+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:42:10.597945+0000","last_undegraded":"2023-07-06T21:42:10.597945+0000","last_fullsized":"2023-07-06T21:42:10.597945+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:14:21.065092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619564+0000","last_change":"2023-07-06T21:42:10.611525+0000","last_active":"2023-07-06T21:42:10.619564+0000","last_peered":"2023-07-06T21:42:10.619564+0000","last_clean":"2023-07-06T21:42:10.619564+0000","last_became_active":"2023-07-06T21:42:10.611211+0000","last_became_peered":"2023-07-06T21:42:10.611211+0000","last_unstale":"2023-07-06T21:42:10.619564+0000","last_undegraded":"2023-07-06T21:42:10.619564+0000","last_fullsized":"2023-07-06T21:42:10.619564+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:45:15.650121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"52'1","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.607722+0000","last_change":"2023-07-06T21:42:10.108755+0000","last_active":"2023-07-06T21:42:10.607722+0000","last_peered":"2023-07-06T21:42:10.607722+0000","last_clean":"2023-07-06T21:42:10.607722+0000","last_became_active":"2023-07-06T21:42:10.108439+0000","last_became_peered":"2023-07-06T21:42:10.108439+0000","last_unstale":"2023-07-06T21:42:10.607722+0000","last_undegraded":"2023-07-06T21:42:10.607722+0000","last_fullsized":"2023-07-06T21:42:10.607722+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:47:12.139600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.601337+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:42:10.601337+0000","last_peered":"2023-07-06T21:42:10.601337+0000","last_clean":"2023-07-06T21:42:10.601337+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:42:10.601337+0000","last_undegraded":"2023-07-06T21:42:10.601337+0000","last_fullsized":"2023-07-06T21:42:10.601337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:17:35.720425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.619671+0000","last_change":"2023-07-06T21:42:10.614243+0000","last_active":"2023-07-06T21:42:10.619671+0000","last_peered":"2023-07-06T21:42:10.619671+0000","last_clean":"2023-07-06T21:42:10.619671+0000","last_became_active":"2023-07-06T21:42:10.613762+0000","last_became_peered":"2023-07-06T21:42:10.613762+0000","last_unstale":"2023-07-06T21:42:10.619671+0000","last_undegraded":"2023-07-06T21:42:10.619671+0000","last_fullsized":"2023-07-06T21:42:10.619671+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:57:25.798752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.590102+0000","last_change":"2023-07-06T21:42:10.107574+0000","last_active":"2023-07-06T21:42:10.590102+0000","last_peered":"2023-07-06T21:42:10.590102+0000","last_clean":"2023-07-06T21:42:10.590102+0000","last_became_active":"2023-07-06T21:42:10.107320+0000","last_became_peered":"2023-07-06T21:42:10.107320+0000","last_unstale":"2023-07-06T21:42:10.590102+0000","last_undegraded":"2023-07-06T21:42:10.590102+0000","last_fullsized":"2023-07-06T21:42:10.590102+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:18:15.795837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.589210+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:42:10.589210+0000","last_peered":"2023-07-06T21:42:10.589210+0000","last_clean":"2023-07-06T21:42:10.589210+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:42:10.589210+0000","last_undegraded":"2023-07-06T21:42:10.589210+0000","last_fullsized":"2023-07-06T21:42:10.589210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:16.679462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.606646+0000","last_change":"2023-07-06T21:42:10.600739+0000","last_active":"2023-07-06T21:42:10.606646+0000","last_peered":"2023-07-06T21:42:10.606646+0000","last_clean":"2023-07-06T21:42:10.606646+0000","last_became_active":"2023-07-06T21:42:10.600485+0000","last_became_peered":"2023-07-06T21:42:10.600485+0000","last_unstale":"2023-07-06T21:42:10.606646+0000","last_undegraded":"2023-07-06T21:42:10.606646+0000","last_fullsized":"2023-07-06T21:42:10.606646+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:10.080335+0000","last_clean_scrub_stamp":"2023-07-06T21:42:10.080335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:37:10.542591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"52'1","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-07-06T21:42:11.591493+0000","last_change":"2023-07-06T21:42:10.108778+0000","last_active":"2023-07-06T21:42:11.591493+0000","last_peered":"2023-07-06T21:42:11.591493+0000","last_clean":"2023-07-06T21:42:11.591493+0000","last_became_active":"2023-07-06T21:42:10.108395+0000","last_became_peered":"2023-07-06T21:42:10.108395+0000","last_unstale":"2023-07-06T21:42:11.591493+0000","last_undegraded":"2023-07-06T21:42:11.591493+0000","last_fullsized":"2023-07-06T21:42:11.591493+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:42:09.070353+0000","last_clean_scrub_stamp":"2023-07-06T21:42:09.070353+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:29:24.120723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-07-06T21:42:10.591120+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:42:10.591120+0000","last_peered":"2023-07-06T21:42:10.591120+0000","last_clean":"2023-07-06T21:42:10.591120+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:42:10.591120+0000","last_undegraded":"2023-07-06T21:42:10.591120+0000","last_fullsized":"2023-07-06T21:42:10.591120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T07:09:00.212444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1192,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1220608,"data_stored":860226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64900000000000002,"15min":0.65700000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85999999999999999,"5min":1.28,"15min":1.28},"last":1.101},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.39800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":0.995,"15min":0.995},"last":0.58099999999999996}]},{"osd":1,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.43099999999999999,"5min":0.371,"15min":0.371},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":1.042},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82599999999999996,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.74399999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69999999999999996,"15min":0.70499999999999996},"min":{"1min":0.49299999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93200000000000005,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.89500000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.70599999999999996,"15min":0.71499999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.83099999999999996,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.85299999999999998,"5min":1.325,"15min":1.325},"last":0.95999999999999996}]},{"osd":4,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72699999999999998,"15min":0.73899999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.86599999999999999,"5min":1.972,"15min":1.972},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.76400000000000001},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0209999999999999,"5min":1.379,"15min":1.379},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.97599999999999998,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.92300000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:41:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.78100000000000003,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96299999999999997,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.81200000000000006,"15min":0.82499999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.996,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.83799999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":618496,"data_stored":269698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73499999999999999,"15min":0.74199999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.21,"5min":3.569,"15min":3.569},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.50900000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79200000000000004,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.79400000000000004,"15min":0.81399999999999995},"min":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.121,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.497}]},{"osd":2,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.79000000000000004,"15min":0.81399999999999995},"min":{"1min":0.441,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.014,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.79100000000000004,"15min":0.81100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92100000000000004,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.032,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.83999999999999997,"15min":0.86499999999999999},"min":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.98499999999999999,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.40000000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65700000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.66500000000000004,"15min":0.65700000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.69699999999999995}]},{"osd":7,"last update":"Thu Jul 6 21:41:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.74399999999999999,"15min":0.746},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.871,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.78800000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":622592,"data_stored":269788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.63700000000000001,"15min":0.65700000000000003},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":3.081,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.6200000000000001,"5min":2.246,"15min":2.246},"last":0.49199999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.56399999999999995,"15min":0.58599999999999997},"min":{"1min":0.32700000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.56599999999999995,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.96499999999999997,"15min":0.97799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":9.6300000000000008,"5min":11.913,"15min":11.913},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":1.262,"15min":1.3600000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.4489999999999998,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.89700000000000002,"15min":0.95099999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.8370000000000002,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.80000000000000004,"15min":0.82099999999999995},"min":{"1min":0.44500000000000001,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89900000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.88800000000000001,"15min":0.89700000000000002},"min":{"1min":0.46000000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":4.2649999999999997,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.72999999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":2.9729999999999999,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.871},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.82299999999999995,"15min":0.84099999999999997},"min":{"1min":0.52100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95899999999999996,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.90500000000000003,"15min":0.90900000000000003},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":4.5540000000000003,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.752,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.7110000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.83199999999999996,"15min":0.82899999999999996},"min":{"1min":0.55800000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.85599999999999998,"15min":0.86799999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.984,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.92100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574887,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28148,"kb_used_data":1196,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343692,"statfs":{"total":96636764160,"available":96607940608,"internally_reserved":0,"allocated":1224704,"data_stored":861193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.51400000000000001,"15min":0.53800000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.54000000000000004,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.52100000000000002,"15min":0.53700000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65100000000000002,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.55300000000000005,"15min":0.57299999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.57799999999999996,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.55600000000000005,"15min":0.57599999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.58999999999999997,"5min":2.29,"15min":2.29},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.48299999999999998,"15min":0.48899999999999999},"min":{"1min":0.373,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70299999999999996,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.60899999999999999,"15min":0.63},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.60799999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.498}]},{"osd":4,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.81200000000000006,"15min":0.83699999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.90300000000000002,"5min":2.234,"15min":2.234},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.84999999999999998},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86599999999999999,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.82999999999999996,"15min":0.85599999999999998},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.82799999999999996,"5min":3.544,"15min":3.544},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82799999999999996},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":2.367,"15min":2.367},"last":0.79900000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.84399999999999997,"15min":0.86599999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.92200000000000004,"5min":2.395,"15min":2.395},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.83099999999999996,"15min":0.85499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.89000000000000001,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.86399999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.84799999999999998,"15min":0.86799999999999999},"min":{"1min":0.54900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.91200000000000003,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.874,"15min":0.90100000000000002},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.89100000000000001,"5min":3.6190000000000002,"15min":3.6190000000000002},"last":0.80800000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":638976,"data_stored":276407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27400000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.435,"15min":0.43099999999999999},"min":{"1min":0.314,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39300000000000002}]},{"osd":1,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.316,"5min":0.315,"15min":0.315},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.36599999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.61599999999999999,"5min":1.369,"15min":1.369},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.61799999999999999,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.48099999999999998,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.57399999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.435}]},{"osd":4,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.745,"15min":0.751},"min":{"1min":0.55600000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.86499999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.746},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.59299999999999997,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83299999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73499999999999999,"15min":0.73899999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.878,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.86599999999999999,"5min":1.476,"15min":1.476},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.78400000000000003,"15min":0.79600000000000004},"min":{"1min":0.58699999999999997,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.86699999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.629},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.91800000000000004,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.83499999999999996,"15min":0.79700000000000004},"min":{"1min":0.58599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":8.3190000000000008,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.61599999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.95999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.78800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":269754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.432,"15min":0.434},"min":{"1min":0.312,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.59299999999999997,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.309},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.437,"15min":0.442},"min":{"1min":0.28799999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.59099999999999997,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.44}]},{"osd":1,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.47599999999999998,"15min":0.47799999999999998},"min":{"1min":0.37,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.36499999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.61399999999999999,"5min":1.155,"15min":1.155},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.59999999999999998,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.88200000000000001,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.51700000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.90800000000000003,"5min":1.304,"15min":1.304},"last":0.55900000000000005}]},{"osd":5,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.77700000000000002,"15min":0.78700000000000003},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.91300000000000003,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.78800000000000003,"15min":0.80100000000000005},"min":{"1min":0.502,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":1.258,"15min":1.258},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.78800000000000003,"15min":0.80800000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.81200000000000006,"5min":2.012,"15min":2.012},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.77200000000000002,"15min":0.78200000000000003},"min":{"1min":0.53800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":0.93100000000000005,"5min":1.29,"15min":1.29},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.77700000000000002,"15min":0.79000000000000004},"min":{"1min":0.54300000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.875,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.76800000000000002,"15min":0.77600000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.89400000000000002,"5min":1.121,"15min":1.121},"last":0.69899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":643072,"data_stored":277915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.64200000000000002,"15min":0.63300000000000001},"min":{"1min":0.48699999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.871,"5min":1.871,"15min":1.871},"last":0.61199999999999999}]},{"osd":1,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.60099999999999998,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.629,"15min":0.621},"min":{"1min":0.45900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66300000000000003,"15min":0.66100000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.67200000000000004,"15min":0.66400000000000003},"min":{"1min":0.53900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.72899999999999998}]},{"osd":3,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.68799999999999994},"min":{"1min":0.498,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.0550000000000002,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.431,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73099999999999998},"min":{"1min":0.48399999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96499999999999997,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.75600000000000001,"15min":0.75700000000000001},"min":{"1min":0.44,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.439,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.73599999999999999,"15min":0.73999999999999999},"min":{"1min":0.46600000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91700000000000004,"5min":1.46,"15min":1.46},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:41:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.75800000000000001,"15min":0.75700000000000001},"min":{"1min":0.42699999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78200000000000003,"15min":0.77900000000000003},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":2.2410000000000001,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.627}]}]},{"osd":5,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27564,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344276,"statfs":{"total":96636764160,"available":96608538624,"internally_reserved":0,"allocated":626688,"data_stored":270350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.66800000000000004,"15min":0.68100000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95299999999999996,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.498},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.68500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89100000000000001,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67800000000000005,"15min":0.68300000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.876,"5min":1.038,"15min":1.038},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66400000000000003,"15min":0.66700000000000004},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.90300000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70599999999999996,"15min":0.71199999999999997},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.628}]},{"osd":3,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69799999999999995,"15min":0.69699999999999995},"min":{"1min":0.58699999999999997,"5min":0.501,"15min":0.501},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72799999999999998,"15min":0.73199999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.94199999999999995,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.40999999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84499999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.497,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.84999999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.71099999999999997,"15min":0.71499999999999997},"min":{"1min":0.495,"5min":0.371,"15min":0.371},"max":{"1min":0.89600000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.751,"15min":0.75700000000000001},"min":{"1min":0.48199999999999998,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85799999999999998,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:41:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77700000000000002,"15min":0.78000000000000003},"min":{"1min":0.57499999999999996,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.90200000000000002,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.96899999999999997,"5min":1.278,"15min":1.278},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:42:15.426 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-06T21:42:15.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-07-06T21:42:15.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.659+0000 7f4fe9dff700 1 Processor -- start 2023-07-06T21:42:15.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.659+0000 7f4fe9dff700 1 -- start start 2023-07-06T21:42:15.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.660+0000 7f4fe9dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4fe40d0ea0 0x7f4fe40d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.660+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe415fee0 0x7f4fe4160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.662 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.660+0000 7f4fe9dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40ceef0 con 0x7f4fe4160ac0 2023-07-06T21:42:15.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.660+0000 7f4fe9dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40cf450 con 0x7f4fe40d0ea0 2023-07-06T21:42:15.663 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.660+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4160860 con 0x7f4fe415fee0 2023-07-06T21:42:15.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe415fee0 0x7f4fe4160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.664 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe37fe700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4fe40d0ea0 0x7f4fe40d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fdbfff700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f4fe4160ac0 0x7f4fe40cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:54850/0 (socket says 172.21.15.84:54850) 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fdbfff700 1 -- 172.21.15.84:0/1644865117 learned_addr learned my addr 172.21.15.84:0/1644865117 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe37fe700 1 -- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe415fee0 msgr2=0x7f4fe4160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe37fe700 1 --2- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe415fee0 0x7f4fe4160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe37fe700 1 -- 172.21.15.84:0/1644865117 >> v1:172.21.15.107:6789/0 conn(0x7f4fe4160ac0 legacy=0x7f4fe40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe37fe700 1 -- 172.21.15.84:0/1644865117 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40cf120 con 0x7f4fe40d0ea0 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.661+0000 7f4fe37fe700 1 --2- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe40d0ea0 0x7f4fe40d1280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009820 tx=0x7f4fd4009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7c84dbf3fd261 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:15.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe2ffd700 1 -- 172.21.15.84:0/1644865117 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4005e30 con 0x7f4fe40d0ea0 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe2ffd700 1 -- 172.21.15.84:0/1644865117 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4006030 con 0x7f4fe40d0ea0 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe2ffd700 1 -- 172.21.15.84:0/1644865117 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40063c0 con 0x7f4fe40d0ea0 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 -- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe40d0ea0 msgr2=0x7f4fe40d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe40d0ea0 0x7f4fe40d1280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009820 tx=0x7f4fd4009b30 comp rx=0 tx=0).stop 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 -- 172.21.15.84:0/1644865117 shutdown_connections 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe415fee0 0x7f4fe4160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/1644865117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe40d0ea0 0x7f4fe40d1280 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 -- 172.21.15.84:0/1644865117 >> 172.21.15.84:0/1644865117 conn(0x7f4fe40715c0 msgr2=0x7f4fe4071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 -- 172.21.15.84:0/1644865117 shutdown_connections 2023-07-06T21:42:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.662+0000 7f4fe9dff700 1 -- 172.21.15.84:0/1644865117 wait complete. 2023-07-06T21:42:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.665+0000 7f4fe9dff700 1 Processor -- start 2023-07-06T21:42:15.667 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.665+0000 7f4fe9dff700 1 -- start start 2023-07-06T21:42:15.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 0x7f4fe41f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe4160ac0 0x7f4fe4152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe9dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe40c8f70 0x7f4fe40c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40cc060 con 0x7f4fe415fee0 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41f5620 con 0x7f4fe4160ac0 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe9dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41f2530 con 0x7f4fe40c8f70 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.666+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 0x7f4fe41f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 0x7f4fe41f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57354/0 (socket says 172.21.15.84:57354) 2023-07-06T21:42:15.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 -- 172.21.15.84:0/353312809 learned_addr learned my addr 172.21.15.84:0/353312809 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:15.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fdbfff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe4160ac0 0x7f4fe4152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe3fff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe40c8f70 0x7f4fe40c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 -- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe40c8f70 msgr2=0x7f4fe40c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe40c8f70 0x7f4fe40c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 -- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe4160ac0 msgr2=0x7f4fe4152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe4160ac0 0x7f4fe4152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 -- 172.21.15.84:0/353312809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd4009430 con 0x7f4fe415fee0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe3fff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe40c8f70 0x7f4fe40c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe37fe700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 0x7f4fe41f1f20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.667+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40063a0 con 0x7f4fe415fee0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.668+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4020070 con 0x7f4fe415fee0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.668+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe41f2810 con 0x7f4fe415fee0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.668+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4006740 con 0x7f4fe415fee0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.668+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe41f2dd0 con 0x7f4fe415fee0 2023-07-06T21:42:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.669+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe404e1d0 con 0x7f4fe415fee0 2023-07-06T21:42:15.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.670+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4fd4022030 con 0x7f4fe415fee0 2023-07-06T21:42:15.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.670+0000 7f4fe0ff9700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fc406ace0 0x7f4fc406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:15.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.671+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4fd408a960 con 0x7f4fe415fee0 2023-07-06T21:42:15.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.671+0000 7f4fdbfff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fc406ace0 0x7f4fc406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:15.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.671+0000 7f4fdbfff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fc406ace0 0x7f4fc406d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4fe41f2460 tx=0x7f4fcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:15.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.675+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4fd4055aa0 con 0x7f4fe415fee0 2023-07-06T21:42:15.949 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.947+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4fe40cde70 con 0x7f4fe415fee0 2023-07-06T21:42:15.955 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.952+0000 7f4fe0ff9700 1 -- 172.21.15.84:0/353312809 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4fd40068a0 con 0x7f4fe415fee0 2023-07-06T21:42:15.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fc406ace0 msgr2=0x7f4fc406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.956 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fc406ace0 0x7f4fc406d190 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4fe41f2460 tx=0x7f4fcc009040 comp rx=0 tx=0).stop 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 msgr2=0x7f4fe41f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 0x7f4fe41f1f20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4000c00 tx=0x7f4fd4004750 comp rx=0 tx=0).stop 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 shutdown_connections 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4fc406ace0 0x7f4fc406d190 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fe40c8f70 0x7f4fe40c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4fe415fee0 0x7f4fe41f1f20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 --2- 172.21.15.84:0/353312809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fe4160ac0 0x7f4fe4152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 >> 172.21.15.84:0/353312809 conn(0x7f4fe40715c0 msgr2=0x7f4fe40cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 shutdown_connections 2023-07-06T21:42:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:15.954+0000 7f4fe9dff700 1 -- 172.21.15.84:0/353312809 wait complete. 2023-07-06T21:42:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:42:15.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:16.169 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.167+0000 7fb1c17ff700 1 Processor -- start 2023-07-06T21:42:16.170 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.167+0000 7fb1c17ff700 1 -- start start 2023-07-06T21:42:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.168+0000 7fb1c17ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1bc0d3580 0x7fb1bc0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.168+0000 7fb1c17ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d13b0 0x7fb1bc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.168+0000 7fb1c17ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1bc0d0990 con 0x7fb1bc0b8530 2023-07-06T21:42:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1c17ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0d3af0 con 0x7fb1bc0d3580 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1c17ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0d1830 con 0x7fb1bc0d13b0 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1bc0d3580 0x7fb1bc0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb1bc0d3580 0x7fb1bc0d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57356/0 (socket says 172.21.15.84:57356) 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 -- 172.21.15.84:0/1477100887 learned_addr learned my addr 172.21.15.84:0/1477100887 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1bb7fe700 1 --2- 172.21.15.84:0/1477100887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d13b0 0x7fb1bc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 -- 172.21.15.84:0/1477100887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d13b0 msgr2=0x7fb1bc15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:16.174 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 --2- 172.21.15.84:0/1477100887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d13b0 0x7fb1bc15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 -- 172.21.15.84:0/1477100887 >> v1:172.21.15.107:6789/0 conn(0x7fb1bc0b8530 legacy=0x7fb1bc0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1b3fff700 1 -- 172.21.15.84:0/1477100887 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0cd9d0 con 0x7fb1bc0d3580 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.169+0000 7fb1ba7fc700 1 -- 172.21.15.84:0/1477100887 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029860711 0 0) 0x7fb1bc0d0990 con 0x7fb1bc0b8530 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1b3fff700 1 --2- 172.21.15.84:0/1477100887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0d3580 0x7fb1bc0d0e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40098f0 tx=0x7fb1a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237f209c506fe8cd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1ba7fc700 1 -- 172.21.15.84:0/1477100887 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4011830 con 0x7fb1bc0d3580 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1ba7fc700 1 -- 172.21.15.84:0/1477100887 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a4011990 con 0x7fb1bc0d3580 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1ba7fc700 1 -- 172.21.15.84:0/1477100887 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a4011d20 con 0x7fb1bc0d3580 2023-07-06T21:42:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1c17ff700 1 -- 172.21.15.84:0/1477100887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0d3580 msgr2=0x7fb1bc0d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/1477100887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0d3580 0x7fb1bc0d0e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40098f0 tx=0x7fb1a4009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1c17ff700 1 -- 172.21.15.84:0/1477100887 shutdown_connections 2023-07-06T21:42:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/1477100887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d13b0 0x7fb1bc15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/1477100887 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0d3580 0x7fb1bc0d0e70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.170+0000 7fb1c17ff700 1 -- 172.21.15.84:0/1477100887 >> 172.21.15.84:0/1477100887 conn(0x7fb1bc0715c0 msgr2=0x7fb1bc071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.171+0000 7fb1c17ff700 1 -- 172.21.15.84:0/1477100887 shutdown_connections 2023-07-06T21:42:16.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.171+0000 7fb1c17ff700 1 -- 172.21.15.84:0/1477100887 wait complete. 2023-07-06T21:42:16.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.172+0000 7fb1c17ff700 1 Processor -- start 2023-07-06T21:42:16.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.172+0000 7fb1c17ff700 1 -- start start 2023-07-06T21:42:16.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1c17ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 0x7fb1bc1f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:16.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1c17ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1bc0d13b0 0x7fb1bc0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1c17ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d3580 0x7fb1bc0c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1c17ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0c9860 con 0x7fb1bc0b8530 2023-07-06T21:42:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1c17ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1f1980 con 0x7fb1bc0d13b0 2023-07-06T21:42:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1c17ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0c9580 con 0x7fb1bc0d3580 2023-07-06T21:42:16.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.173+0000 7fb1baffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 0x7fb1bc1f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 0x7fb1bc1f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57360/0 (socket says 172.21.15.84:57360) 2023-07-06T21:42:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 -- 172.21.15.84:0/2319606923 learned_addr learned my addr 172.21.15.84:0/2319606923 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1bb7fe700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d3580 0x7fb1bc0c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1b3fff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1bc0d13b0 0x7fb1bc0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:16.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 -- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d3580 msgr2=0x7fb1bc0c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d3580 0x7fb1bc0c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 -- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1bc0d13b0 msgr2=0x7fb1bc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1bc0d13b0 0x7fb1bc0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 -- 172.21.15.84:0/2319606923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009550 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.174+0000 7fb1baffd700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 0x7fb1bc1f1440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c550 tx=0x7fb1ac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.175+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac0054e0 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.175+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac0036d0 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.175+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc0c9e90 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.175+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac003a80 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.175+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc0ca450 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.176+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc0c9cd0 con 0x7fb1bc0b8530 2023-07-06T21:42:16.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.177+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb1ac003be0 con 0x7fb1bc0b8530 2023-07-06T21:42:16.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.177+0000 7fb1b37fe700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb19c06ace0 0x7fb19c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:16.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.177+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb1ac0889c0 con 0x7fb1bc0b8530 2023-07-06T21:42:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.178+0000 7fb1b3fff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb19c06ace0 0x7fb19c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.178+0000 7fb1b3fff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb19c06ace0 0x7fb19c06d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4014040 tx=0x7fb1a4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:16.185 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.181+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb1ac0539d0 con 0x7fb1bc0b8530 2023-07-06T21:42:16.418 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.416+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1bc04e240 con 0x7fb1bc0b8530 2023-07-06T21:42:16.419 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.417+0000 7fb1b37fe700 1 -- 172.21.15.84:0/2319606923 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fb1ac017020 con 0x7fb1bc0b8530 2023-07-06T21:42:16.419 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:16.419 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:42:10.578278+0000","modified":"2023-07-06T21:42:13.588255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6834/2341549811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2341549811},{"type":"v1","addr":"172.21.15.107:6834","nonce":2341549811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T21:42:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb19c06ace0 msgr2=0x7fb19c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb19c06ace0 0x7fb19c06d190 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4014040 tx=0x7fb1a4008030 comp rx=0 tx=0).stop 2023-07-06T21:42:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 msgr2=0x7fb1bc1f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:16.421 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 0x7fb1bc1f1440 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c550 tx=0x7fb1ac00c860 comp rx=0 tx=0).stop 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 shutdown_connections 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb19c06ace0 0x7fb19c06d190 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1bc0d3580 0x7fb1bc0c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1bc0b8530 0x7fb1bc1f1440 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 --2- 172.21.15.84:0/2319606923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1bc0d13b0 0x7fb1bc0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 >> 172.21.15.84:0/2319606923 conn(0x7fb1bc0715c0 msgr2=0x7fb1bc160530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:16.422 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 shutdown_connections 2023-07-06T21:42:16.423 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:16.419+0000 7fb1c17ff700 1 -- 172.21.15.84:0/2319606923 wait complete. 2023-07-06T21:42:16.423 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-07-06T21:42:16.844 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-07-06T21:42:10.578278+0000', 'modified': '2023-07-06T21:42:13.588255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6075}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6075': {'gid': 6075, 'name': 'd', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6834/2341549811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6832', 'nonce': 2341549811}, {'type': 'v1', 'addr': '172.21.15.107:6834', 'nonce': 2341549811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-06T21:42:16.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-07-06T21:42:16.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-07-06T21:42:17.043 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.041+0000 7f9b031ff700 1 Processor -- start 2023-07-06T21:42:17.044 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.042+0000 7f9b031ff700 1 -- start start 2023-07-06T21:42:17.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b031ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9afc15c2c0 0x7f9afc0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b031ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c690 0x7f9afc0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b031ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9afc0b4680 0x7f9afc0b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b031ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0b5020 con 0x7f9afc0b4680 2023-07-06T21:42:17.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b031ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc0d1860 con 0x7f9afc15c2c0 2023-07-06T21:42:17.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b031ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0d1580 con 0x7f9afc15c690 2023-07-06T21:42:17.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b0179c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9afc0b4680 0x7f9afc0b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b0179c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f9afc0b4680 0x7f9afc0b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57366/0 (socket says 172.21.15.84:57366) 2023-07-06T21:42:17.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b0179c700 1 -- 172.21.15.84:0/108743624 learned_addr learned my addr 172.21.15.84:0/108743624 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9afbfff700 1 --2- 172.21.15.84:0/108743624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c690 0x7f9afc0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.043+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/108743624 >> v2:172.21.15.107:3300/0 conn(0x7f9afc15c2c0 0x7f9afc0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b0179c700 1 -- 172.21.15.84:0/108743624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c690 msgr2=0x7f9afc0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b0179c700 1 --2- 172.21.15.84:0/108743624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c690 0x7f9afc0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b0179c700 1 -- 172.21.15.84:0/108743624 >> v2:172.21.15.107:3300/0 conn(0x7f9afc15c2c0 msgr2=0x7f9afc0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b0179c700 1 --2- 172.21.15.84:0/108743624 >> v2:172.21.15.107:3300/0 conn(0x7f9afc15c2c0 0x7f9afc0b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b0179c700 1 -- 172.21.15.84:0/108743624 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc0cd380 con 0x7f9afc0b4680 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b0179c700 1 --2- 172.21.15.84:0/108743624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc0b4680 0x7f9afc0b4ae0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0d2b60 tx=0x7f9af400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e945d72fab5143a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9afb7fe700 1 -- 172.21.15.84:0/108743624 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af400e070 con 0x7f9afc0b4680 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9afb7fe700 1 -- 172.21.15.84:0/108743624 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af400b9c0 con 0x7f9afc0b4680 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9afb7fe700 1 -- 172.21.15.84:0/108743624 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af400bd70 con 0x7f9afc0b4680 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b031ff700 1 -- 172.21.15.84:0/108743624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc0b4680 msgr2=0x7f9afc0b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.044+0000 7f9b031ff700 1 --2- 172.21.15.84:0/108743624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc0b4680 0x7f9afc0b4ae0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0d2b60 tx=0x7f9af400b330 comp rx=0 tx=0).stop 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 -- 172.21.15.84:0/108743624 shutdown_connections 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 --2- 172.21.15.84:0/108743624 >> v2:172.21.15.107:3300/0 conn(0x7f9afc15c2c0 0x7f9afc0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 --2- 172.21.15.84:0/108743624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc0b4680 0x7f9afc0b4ae0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 --2- 172.21.15.84:0/108743624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c690 0x7f9afc0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 -- 172.21.15.84:0/108743624 >> 172.21.15.84:0/108743624 conn(0x7f9afc0cf680 msgr2=0x7f9afc0cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 -- 172.21.15.84:0/108743624 shutdown_connections 2023-07-06T21:42:17.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.045+0000 7f9b031ff700 1 -- 172.21.15.84:0/108743624 wait complete. 2023-07-06T21:42:17.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.046+0000 7f9b031ff700 1 Processor -- start 2023-07-06T21:42:17.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.047+0000 7f9b031ff700 1 -- start start 2023-07-06T21:42:17.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.047+0000 7f9b031ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9afc0b4680 0x7f9afc149a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.047+0000 7f9b031ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c2c0 0x7f9afc15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.047+0000 7f9b031ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 0x7f9afc149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b031ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc149590 con 0x7f9afc15c690 2023-07-06T21:42:17.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b031ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc14a0c0 con 0x7f9afc0b4680 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b031ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc14a3a0 con 0x7f9afc15c2c0 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 0x7f9afc149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 0x7f9afc149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57376/0 (socket says 172.21.15.84:57376) 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 -- 172.21.15.84:0/534138571 learned_addr learned my addr 172.21.15.84:0/534138571 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9afc0b4680 0x7f9afc149a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9afbfff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c2c0 0x7f9afc15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 -- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c2c0 msgr2=0x7f9afc15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c2c0 0x7f9afc15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 -- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9afc0b4680 msgr2=0x7f9afc149a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9afc0b4680 0x7f9afc149a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 -- 172.21.15.84:0/534138571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af4009b30 con 0x7f9afc15c690 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.048+0000 7f9b0179c700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 0x7f9afc149050 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0d3d00 tx=0x7f9af4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.049+0000 7f9af97fa700 1 -- 172.21.15.84:0/534138571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af40092e0 con 0x7f9afc15c690 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.049+0000 7f9af97fa700 1 -- 172.21.15.84:0/534138571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4004250 con 0x7f9afc15c690 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.049+0000 7f9af97fa700 1 -- 172.21.15.84:0/534138571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4004620 con 0x7f9afc15c690 2023-07-06T21:42:17.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.049+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc14a680 con 0x7f9afc15c690 2023-07-06T21:42:17.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.049+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc14ac40 con 0x7f9afc15c690 2023-07-06T21:42:17.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.051+0000 7f9af97fa700 1 -- 172.21.15.84:0/534138571 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9af4004780 con 0x7f9afc15c690 2023-07-06T21:42:17.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.052+0000 7f9af97fa700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ae006ace0 0x7f9ae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.052+0000 7f9af97fa700 1 -- 172.21.15.84:0/534138571 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9af408a630 con 0x7f9afc15c690 2023-07-06T21:42:17.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.052+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ae006ace0 0x7f9ae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.053+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ae006ace0 0x7f9ae006d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9af00045f0 tx=0x7f9af0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.054+0000 7f9b16aa8b80 1 Processor -- start 2023-07-06T21:42:17.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.054+0000 7f9b16aa8b80 1 -- start start 2023-07-06T21:42:17.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20afaf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afd6d80 0x55e20afd5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afac740 0x55e20afd9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20aeb3c60 con 0x55e20afaef50 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20af069f0 con 0x55e20afd6d80 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20aeffe80 con 0x55e20afac740 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b00f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20afaf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b00f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20afaf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57384/0 (socket says 172.21.15.84:57384) 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b00f9b700 1 -- 172.21.15.84:0/1533816797 learned_addr learned my addr 172.21.15.84:0/1533816797 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.059 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9b0179c700 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afac740 0x55e20afd9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.055+0000 7f9afbfff700 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afd6d80 0x55e20afd5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b00f9b700 1 -- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afac740 msgr2=0x55e20afd9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afac740 0x55e20afd9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b00f9b700 1 -- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afd6d80 msgr2=0x55e20afd5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afd6d80 0x55e20afd5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b00f9b700 1 -- 172.21.15.84:0/1533816797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20ad815e0 con 0x55e20afaef50 2023-07-06T21:42:17.060 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b0179c700 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afac740 0x55e20afd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:17.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20afaf320 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b750 tx=0x7f9af0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff42120e185a7ba9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9ae67fc700 1 -- 172.21.15.84:0/1533816797 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0016690 con 0x55e20afaef50 2023-07-06T21:42:17.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9ae67fc700 1 -- 172.21.15.84:0/1533816797 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af001a070 con 0x55e20afaef50 2023-07-06T21:42:17.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.056+0000 7f9ae67fc700 1 -- 172.21.15.84:0/1533816797 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af00169f0 con 0x55e20afaef50 2023-07-06T21:42:17.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1533816797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 msgr2=0x55e20afaf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20afaf320 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9af000b750 tx=0x7f9af0013000 comp rx=0 tx=0).stop 2023-07-06T21:42:17.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1533816797 shutdown_connections 2023-07-06T21:42:17.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afac740 0x55e20afd9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20afaf320 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1533816797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afd6d80 0x55e20afd5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1533816797 >> 172.21.15.84:0/1533816797 conn(0x55e20afa3fc0 msgr2=0x55e20aff41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1533816797 shutdown_connections 2023-07-06T21:42:17.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.057+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1533816797 wait complete. 2023-07-06T21:42:17.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.058+0000 7f9b16aa8b80 1 -- start start 2023-07-06T21:42:17.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.058+0000 7f9b16aa8b80 1 Processor -- start 2023-07-06T21:42:17.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afcfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.070 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20aff4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20afa7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20ae9fa70 con 0x55e20afaef50 2023-07-06T21:42:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20ae9f770 con 0x55e20afac740 2023-07-06T21:42:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20ad838d0 con 0x55e20afd6d80 2023-07-06T21:42:17.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b16aa8b80 10 client.0 _ll_register_callbacks cb 0x55e20aea5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T21:42:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20aff4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20aff4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57388/0 (socket says 172.21.15.84:57388) 2023-07-06T21:42:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 -- 172.21.15.84:0/1661465866 learned_addr learned my addr 172.21.15.84:0/1661465866 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afcfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b0179c700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20afa7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 -- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 msgr2=0x55e20afa7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20afa7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 -- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 msgr2=0x55e20afcfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afcfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9afbfff700 1 -- 172.21.15.84:0/1661465866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af000e040 con 0x55e20afaef50 2023-07-06T21:42:17.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.059+0000 7f9b0179c700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20afa7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9afbfff700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20aff4490 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8006b20 tx=0x7f9ae80135c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d4844fe72696d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9adb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9adb7fe700 1 -- 172.21.15.84:0/1661465866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8009690 con 0x55e20afaef50 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9adb7fe700 1 -- 172.21.15.84:0/1661465866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ae8004030 con 0x55e20afaef50 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e20afa1780 con 0x55e20afaef50 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9adb7fe700 1 -- 172.21.15.84:0/1661465866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae80041f0 con 0x55e20afaef50 2023-07-06T21:42:17.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9b16aa8b80 10 client.6219 fetch_fsmap learned FSMap version 24 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e20ad2dd90 con 0x55e20afaef50 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.060+0000 7f9adb7fe700 1 -- 172.21.15.84:0/1661465866 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e20afa1780 con 0x55e20afaef50 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9adb7fe700 1 -- 172.21.15.84:0/1661465866 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9ae8004350 con 0x55e20afaef50 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b16aa8b80 10 client.6219 fetch_fsmap finished waiting for FSMap version 24 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b16aa8b80 10 client.6219 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b16aa8b80 20 client.6219 populate_metadata read hostname 'smithi084' 2023-07-06T21:42:17.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afa84d0 0x55e20afec910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b16aa8b80 4 client.6219 mds_command: new command op to 6075 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 --> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e20aef8e60 con 0x55e20afa84d0 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.061+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afa84d0 0x55e20afec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.062+0000 7f9adb7fe700 10 client.6219 ms_handle_connect on v2:172.21.15.107:6832/2341549811 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.062+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afa84d0 0x55e20afec910 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35f46f2badbf27d server_cookie=d5988083c7dc1dbe in_seq=0 out_seq=0 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9adb7fe700 1 -- 172.21.15.84:0/1661465866 <== mds.0 v2:172.21.15.107:6832/2341549811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e20aef8e60 con 0x55e20afa84d0 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9adb7fe700 10 client.6219 handle_command_reply: tid=0 2023-07-06T21:42:17.077 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 client.6219 shutdown 2023-07-06T21:42:17.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 msgr2=0x55e20aff4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20aff4490 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8006b20 tx=0x7f9ae80135c0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 shutdown_connections 2023-07-06T21:42:17.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afa84d0 0x55e20afec910 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20afa7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.078 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afcfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9adb7fe700 0 client.6219 ms_handle_reset on v2:172.21.15.107:6832/2341549811 2023-07-06T21:42:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/1661465866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afaef50 0x55e20aff4490 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 >> 172.21.15.84:0/1661465866 conn(0x55e20afa3fc0 msgr2=0x55e20afd1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 shutdown_connections 2023-07-06T21:42:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/1661465866 wait complete. 2023-07-06T21:42:17.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.063+0000 7f9b16aa8b80 20 client.6219 trim_cache size 0 max 16384 2023-07-06T21:42:17.083 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.081+0000 7f9b16aa8b80 1 Processor -- start 2023-07-06T21:42:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 -- start start 2023-07-06T21:42:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.084 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afaef50 0x55e20afeb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 0x55e20afeb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20afee4b0 con 0x55e20afd6d80 2023-07-06T21:42:17.085 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20afa1360 con 0x55e20afac740 2023-07-06T21:42:17.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.082+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20afd0010 con 0x55e20afaef50 2023-07-06T21:42:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 0x55e20afeb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 0x55e20afeb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57392/0 (socket says 172.21.15.84:57392) 2023-07-06T21:42:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 -- 172.21.15.84:0/942587880 learned_addr learned my addr 172.21.15.84:0/942587880 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9afbfff700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afaef50 0x55e20afeb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 -- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afaef50 msgr2=0x55e20afeb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afaef50 0x55e20afeb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 -- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 msgr2=0x55e20afae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 -- 172.21.15.84:0/942587880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20ad16c10 con 0x55e20afd6d80 2023-07-06T21:42:17.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9afbfff700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afaef50 0x55e20afeb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.083+0000 7f9b0179c700 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 0x55e20afeb6d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b370 tx=0x7f9af4058fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538f2b2d6a7c1ff6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9ae6ffd700 1 -- 172.21.15.84:0/942587880 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4070070 con 0x55e20afd6d80 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9ae6ffd700 1 -- 172.21.15.84:0/942587880 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af4057730 con 0x55e20afd6d80 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9ae6ffd700 1 -- 172.21.15.84:0/942587880 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af4055aa0 con 0x55e20afd6d80 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/942587880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 msgr2=0x55e20afeb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 0x55e20afeb6d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b370 tx=0x7f9af4058fa0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/942587880 shutdown_connections 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afd6d80 0x55e20afeb6d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afac740 0x55e20afae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/942587880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afaef50 0x55e20afeb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/942587880 >> 172.21.15.84:0/942587880 conn(0x55e20afa3fc0 msgr2=0x55e20b0008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/942587880 shutdown_connections 2023-07-06T21:42:17.090 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.084+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/942587880 wait complete. 2023-07-06T21:42:17.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.085+0000 7f9b16aa8b80 1 -- start start 2023-07-06T21:42:17.091 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.085+0000 7f9b16aa8b80 1 Processor -- start 2023-07-06T21:42:17.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 0x55e20afd9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afaef50 0x55e20b003090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20b0035d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20afda160 con 0x55e20afac740 2023-07-06T21:42:17.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20afd6580 con 0x55e20afaef50 2023-07-06T21:42:17.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20afd9e80 con 0x55e20afd6d80 2023-07-06T21:42:17.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b16aa8b80 10 client.0 _ll_register_callbacks cb 0x55e20aea5210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-06T21:42:17.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b00f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 0x55e20afd9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b00f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 0x55e20afd9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57406/0 (socket says 172.21.15.84:57406) 2023-07-06T21:42:17.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.086+0000 7f9b00f9b700 1 -- 172.21.15.84:0/4169691970 learned_addr learned my addr 172.21.15.84:0/4169691970 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9afbfff700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afaef50 0x55e20b003090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b0179c700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20b0035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b00f9b700 1 -- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 msgr2=0x55e20b0035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20b0035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b00f9b700 1 -- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afaef50 msgr2=0x55e20b003090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afaef50 0x55e20b003090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b00f9b700 1 -- 172.21.15.84:0/4169691970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc14ac40 con 0x55e20afac740 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9afbfff700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afaef50 0x55e20b003090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:17.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b0179c700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20b0035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b00f9b700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 0x55e20afd9520 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9af0023040 tx=0x7f9af0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b72824627c530b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9adb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9adb7fe700 1 -- 172.21.15.84:0/4169691970 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af000a070 con 0x55e20afac740 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9af001a070 con 0x55e20afac740 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9adb7fe700 1 -- 172.21.15.84:0/4169691970 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9af0033540 con 0x55e20afac740 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9adb7fe700 1 -- 172.21.15.84:0/4169691970 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af00337f0 con 0x55e20afac740 2023-07-06T21:42:17.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b16aa8b80 10 client.6225 fetch_fsmap learned FSMap version 24 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.087+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e20ad54200 con 0x55e20afac740 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9adb7fe700 1 -- 172.21.15.84:0/4169691970 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af001a070 con 0x55e20afac740 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9adb7fe700 1 -- 172.21.15.84:0/4169691970 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9af0033950 con 0x55e20afac740 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9b16aa8b80 10 client.6225 fetch_fsmap finished waiting for FSMap version 24 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9b16aa8b80 10 client.6225 resolve_mds: resolved name 'd' to daemon mds.d 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9b16aa8b80 20 client.6225 populate_metadata read hostname 'smithi084' 2023-07-06T21:42:17.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afd5800 0x55e20afd5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9b16aa8b80 4 client.6225 mds_command: new command op to 6075 tid=0[{"prefix": "status"}] 2023-07-06T21:42:17.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 --> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e20aebb000 con 0x55e20afd5800 2023-07-06T21:42:17.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.088+0000 7f9afbfff700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afd5800 0x55e20afd5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9adb7fe700 10 client.6225 ms_handle_connect on v2:172.21.15.107:6832/2341549811 2023-07-06T21:42:17.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9afbfff700 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afd5800 0x55e20afd5bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50935ca33301895d server_cookie=27f4e0fedbebf159 in_seq=0 out_seq=0 2023-07-06T21:42:17.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9adb7fe700 1 -- 172.21.15.84:0/4169691970 <== mds.0 v2:172.21.15.107:6832/2341549811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e20aebb000 con 0x55e20afd5800 2023-07-06T21:42:17.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9adb7fe700 10 client.6225 handle_command_reply: tid=0 2023-07-06T21:42:17.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 client.6225 shutdown 2023-07-06T21:42:17.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 msgr2=0x55e20afd9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 0x55e20afd9520 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9af0023040 tx=0x7f9af0017000 comp rx=0 tx=0).stop 2023-07-06T21:42:17.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 shutdown_connections 2023-07-06T21:42:17.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:6832/2341549811,v1:172.21.15.107:6834/2341549811] conn(0x55e20afd5800 0x55e20afd5bd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e20afd6d80 0x55e20b0035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e20afac740 0x55e20afd9520 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9adb7fe700 0 client.6225 ms_handle_reset on v2:172.21.15.107:6832/2341549811 2023-07-06T21:42:17.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 --2- 172.21.15.84:0/4169691970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e20afaef50 0x55e20b003090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 >> 172.21.15.84:0/4169691970 conn(0x55e20afa3fc0 msgr2=0x55e20afae2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 shutdown_connections 2023-07-06T21:42:17.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.089+0000 7f9b16aa8b80 1 -- 172.21.15.84:0/4169691970 wait complete. 2023-07-06T21:42:17.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b16aa8b80 20 client.6225 trim_cache size 0 max 16384 2023-07-06T21:42:17.104 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:42:17.105 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "c2762452-5842-4ccc-b0b9-8dbf6ac718c9", 2023-07-06T21:42:17.105 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2023-07-06T21:42:17.105 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6075, 2023-07-06T21:42:17.106 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2023-07-06T21:42:17.106 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2023-07-06T21:42:17.106 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2023-07-06T21:42:17.107 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 6.5084225910000004, 2023-07-06T21:42:17.107 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 24, 2023-07-06T21:42:17.107 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 53, 2023-07-06T21:42:17.108 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 53, 2023-07-06T21:42:17.108 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 12.240288109 2023-07-06T21:42:17.108 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:42:17.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ae006ace0 msgr2=0x7f9ae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ae006ace0 0x7f9ae006d190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9af00045f0 tx=0x7f9af0008040 comp rx=0 tx=0).stop 2023-07-06T21:42:17.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 msgr2=0x7f9afc149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 0x7f9afc149050 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0d3d00 tx=0x7f9af4004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 shutdown_connections 2023-07-06T21:42:17.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9ae006ace0 0x7f9ae006d190 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9afc15c690 0x7f9afc149050 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9afc0b4680 0x7f9afc149a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 --2- 172.21.15.84:0/534138571 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9afc15c2c0 0x7f9afc15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 >> 172.21.15.84:0/534138571 conn(0x7f9afc0cf680 msgr2=0x7f9afc15a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 shutdown_connections 2023-07-06T21:42:17.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.090+0000 7f9b031ff700 1 -- 172.21.15.84:0/534138571 wait complete. 2023-07-06T21:42:17.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:42:17.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.305+0000 7fb5ccbff700 1 Processor -- start 2023-07-06T21:42:17.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.305+0000 7fb5ccbff700 1 -- start start 2023-07-06T21:42:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.306+0000 7fb5ccbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c8154910 0x7fb5c8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.306+0000 7fb5ccbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5c815a7e0 0x7fb5c815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.306+0000 7fb5ccbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c8070230 con 0x7fb5c80c8d00 2023-07-06T21:42:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.306+0000 7fb5ccbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c815b2a0 con 0x7fb5c815a7e0 2023-07-06T21:42:17.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.306+0000 7fb5ccbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c815b580 con 0x7fb5c8154910 2023-07-06T21:42:17.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5c815a7e0 0x7fb5c815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb5c815a7e0 0x7fb5c815ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57416/0 (socket says 172.21.15.84:57416) 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/86727263 learned_addr learned my addr 172.21.15.84:0/86727263 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c659c700 1 --2- 172.21.15.84:0/86727263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c8154910 0x7fb5c8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/86727263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c8154910 msgr2=0x7fb5c8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 --2- 172.21.15.84:0/86727263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c8154910 0x7fb5c8070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/86727263 >> v1:172.21.15.107:6789/0 conn(0x7fb5c80c8d00 legacy=0x7fb5c80c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/86727263 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8070480 con 0x7fb5c815a7e0 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.307+0000 7fb5c5d9b700 1 --2- 172.21.15.84:0/86727263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c815a7e0 0x7fb5c815ac50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0098f0 tx=0x7fb5bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74042d29ac5309be server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5c559a700 1 -- 172.21.15.84:0/86727263 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc011830 con 0x7fb5c815a7e0 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5c559a700 1 -- 172.21.15.84:0/86727263 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc011990 con 0x7fb5c815a7e0 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5c559a700 1 -- 172.21.15.84:0/86727263 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc011d20 con 0x7fb5c815a7e0 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 -- 172.21.15.84:0/86727263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c815a7e0 msgr2=0x7fb5c815ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/86727263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c815a7e0 0x7fb5c815ac50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0098f0 tx=0x7fb5bc009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:17.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 -- 172.21.15.84:0/86727263 shutdown_connections 2023-07-06T21:42:17.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/86727263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c8154910 0x7fb5c8070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/86727263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c815a7e0 0x7fb5c815ac50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 -- 172.21.15.84:0/86727263 >> 172.21.15.84:0/86727263 conn(0x7fb5c80cf3d0 msgr2=0x7fb5c80cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 -- 172.21.15.84:0/86727263 shutdown_connections 2023-07-06T21:42:17.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.308+0000 7fb5ccbff700 1 -- 172.21.15.84:0/86727263 wait complete. 2023-07-06T21:42:17.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.310+0000 7fb5ccbff700 1 Processor -- start 2023-07-06T21:42:17.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.310+0000 7fb5ccbff700 1 -- start start 2023-07-06T21:42:17.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5ccbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c80c8d00 0x7fb5c814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5ccbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 0x7fb5c80aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5ccbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c815a7e0 0x7fb5c80af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5ccbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c6be0 con 0x7fb5c8154910 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5ccbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80b0fe0 con 0x7fb5c815a7e0 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5ccbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c80c6870 con 0x7fb5c80c8d00 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5c5d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 0x7fb5c80aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5c5d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 0x7fb5c80aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57430/0 (socket says 172.21.15.84:57430) 2023-07-06T21:42:17.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.311+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/1507019178 learned_addr learned my addr 172.21.15.84:0/1507019178 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c659c700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c80c8d00 0x7fb5c814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c6d9d700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c815a7e0 0x7fb5c80af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c80c8d00 msgr2=0x7fb5c814f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c5d9b700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c80c8d00 0x7fb5c814f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c815a7e0 msgr2=0x7fb5c80af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c5d9b700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c815a7e0 0x7fb5c80af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c5d9b700 1 -- 172.21.15.84:0/1507019178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc009550 con 0x7fb5c8154910 2023-07-06T21:42:17.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5c5d9b700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 0x7fb5c80aec80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009c40 tx=0x7fb5bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc0036a0 con 0x7fb5c8154910 2023-07-06T21:42:17.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc01e070 con 0x7fb5c8154910 2023-07-06T21:42:17.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc003a60 con 0x7fb5c8154910 2023-07-06T21:42:17.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.312+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c814f800 con 0x7fb5c8154910 2023-07-06T21:42:17.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.313+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c81501c0 con 0x7fb5c8154910 2023-07-06T21:42:17.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.313+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c80c66d0 con 0x7fb5c8154910 2023-07-06T21:42:17.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.314+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb5bc003bc0 con 0x7fb5c8154910 2023-07-06T21:42:17.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.315+0000 7fb5b6ffd700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb5a806ace0 0x7fb5a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:17.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.315+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb5bc08a3d0 con 0x7fb5c8154910 2023-07-06T21:42:17.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.315+0000 7fb5c659c700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb5a806ace0 0x7fb5a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:17.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.316+0000 7fb5c659c700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb5a806ace0 0x7fb5a806d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009b70 tx=0x7fb5b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:17.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.319+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb5c80c66d0 con 0x7fb5c8154910 2023-07-06T21:42:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.559+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5c80c66d0 con 0x7fb5c8154910 2023-07-06T21:42:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.559+0000 7fb5b6ffd700 1 -- 172.21.15.84:0/1507019178 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb5c80c66d0 con 0x7fb5c8154910 2023-07-06T21:42:17.561 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:42:17.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb5a806ace0 msgr2=0x7fb5a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb5a806ace0 0x7fb5a806d190 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009b70 tx=0x7fb5b8008040 comp rx=0 tx=0).stop 2023-07-06T21:42:17.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 msgr2=0x7fb5c80aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:17.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 0x7fb5c80aec80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009c40 tx=0x7fb5bc004750 comp rx=0 tx=0).stop 2023-07-06T21:42:17.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 shutdown_connections 2023-07-06T21:42:17.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb5a806ace0 0x7fb5a806d190 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5c815a7e0 0x7fb5c80af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5c80c8d00 0x7fb5c814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 --2- 172.21.15.84:0/1507019178 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb5c8154910 0x7fb5c80aec80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 >> 172.21.15.84:0/1507019178 conn(0x7fb5c80cf3d0 msgr2=0x7fb5c815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.561+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 shutdown_connections 2023-07-06T21:42:17.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:17.562+0000 7fb5ccbff700 1 -- 172.21.15.84:0/1507019178 wait complete. 2023-07-06T21:42:17.582 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:42:17.583 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:42:17.583 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:42:17.584 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:42:17.585 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:17.585 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:42:17.586 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:42:17.586 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:42:17.587 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:17.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:17.589 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:42:17.632 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:42:17.632 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:42:17.633 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:42:17.633 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.633 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:42:17.633 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.633 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:17.634 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:17.634 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:42:17.634 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:17.634 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:17.634 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:42:17.634 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42064sec preferred_lft 42064sec 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:42:17.635 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:42:17.636 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:42:17.637 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:42:17.637 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.637 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:42:17.637 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:42:17.637 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4492:bbff:fe16:eebb/64 scope link 2023-07-06T21:42:17.637 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:17.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:17.638 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:42:17.738 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:42:17.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:17.740 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:42:17.787 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:42:17.787 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:42:17.787 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:42:17.787 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:42:17.787 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:42:17.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:17.788 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:42:17.789 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:42:17.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:42:17.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:42:17.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:42:17.790 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:42:17.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:17.995 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:42:18.037 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:42:18.038 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:42:18.039 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:18.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:18.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:42:18.146 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:42:18.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:42:18.147 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:18.189 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:18.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:42:18.190 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:18.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:42:18.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:42:18.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:42:18.388 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:42:18.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:42:18.512 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:42:18.515 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:42:18.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:18.517 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:42:18.561 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:42:18.561 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:42:18.607 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:42:18.678 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:42:18.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:42:18.747 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:42:18.751 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:42:18.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:18.752 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:42:18.779 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:42:18.777+0000 7fdcd84d55c0 -1 init, newargv = 0x564a3c8f2b50 newargc=15 2023-07-06T21:42:18.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[153719]: starting ceph client 2023-07-06T21:42:18.787 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[153719]: starting fuse 2023-07-06T21:42:18.799 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:42:18.799 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:42:18.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:42:18.800 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:42:18.800 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:42:18.801 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:42:18.801 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:42:18.801 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:18.802 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:42:18.802 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:42:18.803 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:42:18.803 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:42:18.803 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:18.804 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:42:18.804 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:42:18.804 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:42:18.805 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:42:18.805 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:18.805 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:42:18.806 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:42:18.806 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:42:18.806 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:42:18.807 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:42:18.807 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:42:18.807 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:42:18.808 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:42:18.808 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:18.809 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:42:18.810 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:42:18.980 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.153719.asok 2023-07-06T21:42:19.158 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.153719.asok 2023-07-06T21:42:19.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:19.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.153719.asok status 2023-07-06T21:42:19.380 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:42:19.380 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:42:19.380 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:42:19.380 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:42:19.380 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:42:19.380 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "153719", 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:42:19.381 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6237, 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 6237 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:42:19.382 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 239798719 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:42:19.383 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 239798719 2023-07-06T21:42:19.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:19.384 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.6237 192.168.0.1:0/239798719", 2023-07-06T21:42:19.384 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/239798719", 2023-07-06T21:42:19.384 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:42:19.384 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 24, 2023-07-06T21:42:19.385 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 53, 2023-07-06T21:42:19.385 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:42:19.385 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:42:19.385 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:42:19.385 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:42:19.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:19.395 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:42:19.396 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:19.439 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:42:19.439 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:19.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:19.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:19.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:19.548 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2023-07-06T21:42:19.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2023-07-06T21:42:19.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.798+0000 7f422dfff700 1 Processor -- start 2023-07-06T21:42:19.808 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.798+0000 7f422dfff700 1 -- start start 2023-07-06T21:42:19.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f422dfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4228160ac0 0x7f42280cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:19.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f422dfff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42280ceef0 con 0x7f42280d0ea0 2023-07-06T21:42:19.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f422dfff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42280cf450 con 0x7f42280cfe20 2023-07-06T21:42:19.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f422dfff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42280cf120 con 0x7f4228160ac0 2023-07-06T21:42:19.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f4226ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f4228160ac0 0x7f42280cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:19.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f42277fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f42280d0ea0 0x7f42280d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56198/0 (socket says 172.21.15.84:56198) 2023-07-06T21:42:19.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.799+0000 7f42277fe700 1 -- 172.21.15.84:0/3457481388 learned_addr learned my addr 172.21.15.84:0/3457481388 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:19.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f4226ffd700 1 -- 172.21.15.84:0/3457481388 >> v1:172.21.15.107:6789/0 conn(0x7f42280cfe20 legacy=0x7f422815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:19.817 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f4226ffd700 1 -- 172.21.15.84:0/3457481388 >> v1:172.21.15.84:6789/0 conn(0x7f42280d0ea0 legacy=0x7f42280d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f4226ffd700 1 -- 172.21.15.84:0/3457481388 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280cd420 con 0x7f4228160ac0 2023-07-06T21:42:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f42267fc700 1 -- 172.21.15.84:0/3457481388 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027392881 0 0) 0x7f42280ceef0 con 0x7f42280d0ea0 2023-07-06T21:42:19.818 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f4226ffd700 1 --2- 172.21.15.84:0/3457481388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4228160ac0 0x7f42280cf8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f421c009840 tx=0x7f421c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da3c988017d229 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f42267fc700 1 -- 172.21.15.84:0/3457481388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c004230 con 0x7f4228160ac0 2023-07-06T21:42:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f42267fc700 1 -- 172.21.15.84:0/3457481388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c004390 con 0x7f4228160ac0 2023-07-06T21:42:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.800+0000 7f42267fc700 1 -- 172.21.15.84:0/3457481388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421c0036a0 con 0x7f4228160ac0 2023-07-06T21:42:19.819 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 -- 172.21.15.84:0/3457481388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4228160ac0 msgr2=0x7f42280cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 --2- 172.21.15.84:0/3457481388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4228160ac0 0x7f42280cf8e0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f421c009840 tx=0x7f421c009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 -- 172.21.15.84:0/3457481388 shutdown_connections 2023-07-06T21:42:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 --2- 172.21.15.84:0/3457481388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4228160ac0 0x7f42280cf8e0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 -- 172.21.15.84:0/3457481388 >> 172.21.15.84:0/3457481388 conn(0x7f42280715c0 msgr2=0x7f4228071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:19.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 -- 172.21.15.84:0/3457481388 shutdown_connections 2023-07-06T21:42:19.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.801+0000 7f422dfff700 1 -- 172.21.15.84:0/3457481388 wait complete. 2023-07-06T21:42:19.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.803+0000 7f422dfff700 1 Processor -- start 2023-07-06T21:42:19.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.803+0000 7f422dfff700 1 -- start start 2023-07-06T21:42:19.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f422dfff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42280cfe20 0x7f4228153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:19.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f422dfff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4228160ac0 0x7f42280cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:19.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f422dfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 0x7f42280cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:19.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f422dfff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228154830 con 0x7f42281543a0 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f422dfff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280b7150 con 0x7f42280cfe20 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f422dfff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42280b7430 con 0x7f4228160ac0 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 0x7f42280cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 0x7f42280cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57440/0 (socket says 172.21.15.84:57440) 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f42277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42280cfe20 0x7f4228153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 -- 172.21.15.84:0/302042300 learned_addr learned my addr 172.21.15.84:0/302042300 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:19.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4226ffd700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4228160ac0 0x7f42280cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:19.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 -- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4228160ac0 msgr2=0x7f42280cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:19.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4228160ac0 0x7f42280cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:19.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 -- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42280cfe20 msgr2=0x7f4228153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:19.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.804+0000 7f4227fff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42280cfe20 0x7f4228153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:19.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f4227fff700 1 -- 172.21.15.84:0/302042300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c0094a0 con 0x7f42281543a0 2023-07-06T21:42:19.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f4227fff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 0x7f42280cba90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f422000cba0 tx=0x7f422000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000aaa0 con 0x7f42281543a0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422000ac00 con 0x7f42281543a0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4220012420 con 0x7f42281543a0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280b7710 con 0x7f42281543a0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.805+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4228153f50 con 0x7f42281543a0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.806+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42280b7900 con 0x7f42281543a0 2023-07-06T21:42:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.807+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4220012580 con 0x7f42281543a0 2023-07-06T21:42:19.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.808+0000 7f420ffff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f421406ace0 0x7f421406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:19.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.808+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4220089a70 con 0x7f42281543a0 2023-07-06T21:42:19.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.808+0000 7f42277fe700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f421406ace0 0x7f421406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:19.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.809+0000 7f42277fe700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f421406ace0 0x7f421406d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f42280d1bc0 tx=0x7f4218009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:19.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:19.813+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4220054b10 con 0x7f42281543a0 2023-07-06T21:42:20.089 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.087+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) v1 -- 0x7f422804e1d0 con 0x7f42281543a0 2023-07-06T21:42:20.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.091+0000 7f420ffff700 1 -- 172.21.15.84:0/302042300 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 183+0+0 (secure 0 0 0) 0x7f4220016020 con 0x7f42281543a0 2023-07-06T21:42:20.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f421406ace0 msgr2=0x7f421406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f421406ace0 0x7f421406d190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f42280d1bc0 tx=0x7f4218009040 comp rx=0 tx=0).stop 2023-07-06T21:42:20.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 msgr2=0x7f42280cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 0x7f42280cba90 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f422000cba0 tx=0x7f422000ceb0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 shutdown_connections 2023-07-06T21:42:20.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f421406ace0 0x7f421406d190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f42281543a0 0x7f42280cba90 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42280cfe20 0x7f4228153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 --2- 172.21.15.84:0/302042300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4228160ac0 0x7f42280cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 >> 172.21.15.84:0/302042300 conn(0x7f42280715c0 msgr2=0x7f42280c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:20.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 shutdown_connections 2023-07-06T21:42:20.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.094+0000 7f422dfff700 1 -- 172.21.15.84:0/302042300 wait complete. 2023-07-06T21:42:20.100 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:42:20.114 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:20.114 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:20.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.388+0000 7fe462fff700 1 Processor -- start 2023-07-06T21:42:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.388+0000 7fe462fff700 1 -- start start 2023-07-06T21:42:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.389+0000 7fe462fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe45c0b83c0 0x7fe45c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.389+0000 7fe462fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0cfc60 0x7fe45c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.389+0000 7fe462fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45c0d0a80 con 0x7fe45c0d3710 2023-07-06T21:42:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.389+0000 7fe462fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45c0b8790 con 0x7fe45c0b83c0 2023-07-06T21:42:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.389+0000 7fe462fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0b4e70 con 0x7fe45c0cfc60 2023-07-06T21:42:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.389+0000 7fe460d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe45c0b83c0 0x7fe45c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe45c0b83c0 0x7fe45c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57452/0 (socket says 172.21.15.84:57452) 2023-07-06T21:42:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 -- 172.21.15.84:0/3630573577 learned_addr learned my addr 172.21.15.84:0/3630573577 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe45bfff700 1 --1- 172.21.15.84:0/3630573577 >> v1:172.21.15.84:6789/0 conn(0x7fe45c0d3710 0x7fe45c0cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56204/0 (socket says 172.21.15.84:56204) 2023-07-06T21:42:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe46159c700 1 --2- 172.21.15.84:0/3630573577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0cfc60 0x7fe45c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 -- 172.21.15.84:0/3630573577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0cfc60 msgr2=0x7fe45c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:20.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 --2- 172.21.15.84:0/3630573577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0cfc60 0x7fe45c0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 -- 172.21.15.84:0/3630573577 >> v1:172.21.15.84:6789/0 conn(0x7fe45c0d3710 legacy=0x7fe45c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 -- 172.21.15.84:0/3630573577 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c0b5150 con 0x7fe45c0b83c0 2023-07-06T21:42:20.394 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe460d9b700 1 --2- 172.21.15.84:0/3630573577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0b83c0 0x7fe45c0cd200 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe4500098f0 tx=0x7fe450009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f27568202a4df5e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe45b7fe700 1 -- 172.21.15.84:0/3630573577 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000b8a0 con 0x7fe45c0b83c0 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe45b7fe700 1 -- 172.21.15.84:0/3630573577 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45000ba00 con 0x7fe45c0b83c0 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.390+0000 7fe45b7fe700 1 -- 172.21.15.84:0/3630573577 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000bd90 con 0x7fe45c0b83c0 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 -- 172.21.15.84:0/3630573577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0b83c0 msgr2=0x7fe45c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 --2- 172.21.15.84:0/3630573577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0b83c0 0x7fe45c0cd200 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe4500098f0 tx=0x7fe450009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 -- 172.21.15.84:0/3630573577 shutdown_connections 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 --2- 172.21.15.84:0/3630573577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0cfc60 0x7fe45c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.395 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 --2- 172.21.15.84:0/3630573577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0b83c0 0x7fe45c0cd200 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 -- 172.21.15.84:0/3630573577 >> 172.21.15.84:0/3630573577 conn(0x7fe45c06b350 msgr2=0x7fe45c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 -- 172.21.15.84:0/3630573577 shutdown_connections 2023-07-06T21:42:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.391+0000 7fe462fff700 1 -- 172.21.15.84:0/3630573577 wait complete. 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.393+0000 7fe462fff700 1 Processor -- start 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.393+0000 7fe462fff700 1 -- start start 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe462fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0b83c0 0x7fe45c150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe462fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 0x7fe45c0cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe462fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe45c0d3710 0x7fe45c0cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe462fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0cb510 con 0x7fe45c0cfc60 2023-07-06T21:42:20.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe462fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0ca620 con 0x7fe45c0d3710 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe462fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0ca900 con 0x7fe45c0b83c0 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe45bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 0x7fe45c0cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe45bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 0x7fe45c0cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57458/0 (socket says 172.21.15.84:57458) 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe45bfff700 1 -- 172.21.15.84:0/2817872475 learned_addr learned my addr 172.21.15.84:0/2817872475 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe460d9b700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0b83c0 0x7fe45c150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.394+0000 7fe46159c700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe45c0d3710 0x7fe45c0cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe45bfff700 1 -- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0b83c0 msgr2=0x7fe45c150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe45bfff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0b83c0 0x7fe45c150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe45bfff700 1 -- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe45c0d3710 msgr2=0x7fe45c0cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe45bfff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe45c0d3710 0x7fe45c0cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe45bfff700 1 -- 172.21.15.84:0/2817872475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450009550 con 0x7fe45c0cfc60 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe45bfff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 0x7fe45c0cc000 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c4f0 tx=0x7fe44800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448005cb0 con 0x7fe45c0cfc60 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe448005e10 con 0x7fe45c0cfc60 2023-07-06T21:42:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c0cabe0 con 0x7fe45c0cfc60 2023-07-06T21:42:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480072d0 con 0x7fe45c0cfc60 2023-07-06T21:42:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.395+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c0cb1a0 con 0x7fe45c0cfc60 2023-07-06T21:42:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.397+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c0d1f20 con 0x7fe45c0cfc60 2023-07-06T21:42:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.397+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe448007430 con 0x7fe45c0cfc60 2023-07-06T21:42:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.399+0000 7fe4597fa700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe44006ace0 0x7fe44006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.399+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4480886e0 con 0x7fe45c0cfc60 2023-07-06T21:42:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.399+0000 7fe460d9b700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe44006ace0 0x7fe44006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.400+0000 7fe460d9b700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe44006ace0 0x7fe44006d190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe450014040 tx=0x7fe450008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.403+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe448053840 con 0x7fe45c0cfc60 2023-07-06T21:42:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.598+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe45c0d2470 con 0x7fe45c0cfc60 2023-07-06T21:42:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.600+0000 7fe4597fa700 1 -- 172.21.15.84:0/2817872475 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7fe448056e60 con 0x7fe45c0cfc60 2023-07-06T21:42:20.602 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:20.602 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:20.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe44006ace0 msgr2=0x7fe44006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.604 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe44006ace0 0x7fe44006d190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe450014040 tx=0x7fe450008030 comp rx=0 tx=0).stop 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 msgr2=0x7fe45c0cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 0x7fe45c0cc000 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c4f0 tx=0x7fe44800c800 comp rx=0 tx=0).stop 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 shutdown_connections 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe44006ace0 0x7fe44006d190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe45c0d3710 0x7fe45c0cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe45c0b83c0 0x7fe45c150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 --2- 172.21.15.84:0/2817872475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe45c0cfc60 0x7fe45c0cc000 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 >> 172.21.15.84:0/2817872475 conn(0x7fe45c06b350 msgr2=0x7fe45c15aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 shutdown_connections 2023-07-06T21:42:20.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.602+0000 7fe462fff700 1 -- 172.21.15.84:0/2817872475 wait complete. 2023-07-06T21:42:20.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:42:08.774391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:42:09.558402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:20.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:20.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.818+0000 7fcca9bff700 1 Processor -- start 2023-07-06T21:42:20.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.818+0000 7fcca9bff700 1 -- start start 2023-07-06T21:42:20.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca9bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcca40b81c0 0x7fcca40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca9bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcca40ceda0 0x7fcca40cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca9bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca40d1830 con 0x7fcca40b1a80 2023-07-06T21:42:20.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca9bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca40cf330 con 0x7fcca40b81c0 2023-07-06T21:42:20.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca9bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca40d1270 con 0x7fcca40ceda0 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca37fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcca40b81c0 0x7fcca40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca37fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fcca40b81c0 0x7fcca40cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57460/0 (socket says 172.21.15.84:57460) 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.819+0000 7fcca37fe700 1 -- 172.21.15.84:0/340120621 learned_addr learned my addr 172.21.15.84:0/340120621 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca3fff700 1 --2- 172.21.15.84:0/340120621 >> v2:172.21.15.107:3300/0 conn(0x7fcca40ceda0 0x7fcca40cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca37fe700 1 -- 172.21.15.84:0/340120621 >> v2:172.21.15.107:3300/0 conn(0x7fcca40ceda0 msgr2=0x7fcca40cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca37fe700 1 --2- 172.21.15.84:0/340120621 >> v2:172.21.15.107:3300/0 conn(0x7fcca40ceda0 0x7fcca40cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca37fe700 1 -- 172.21.15.84:0/340120621 >> v1:172.21.15.107:6789/0 conn(0x7fcca40b1a80 legacy=0x7fcca415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca37fe700 1 -- 172.21.15.84:0/340120621 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca40d1550 con 0x7fcca40b81c0 2023-07-06T21:42:20.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca37fe700 1 --2- 172.21.15.84:0/340120621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40b81c0 0x7fcca40cf600 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcca40c9c50 tx=0x7fcc94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234c12d0a19ec946 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:20.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca2ffd700 1 -- 172.21.15.84:0/340120621 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94011830 con 0x7fcca40b81c0 2023-07-06T21:42:20.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca2ffd700 1 -- 172.21.15.84:0/340120621 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc94011990 con 0x7fcca40b81c0 2023-07-06T21:42:20.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.820+0000 7fcca2ffd700 1 -- 172.21.15.84:0/340120621 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94011d20 con 0x7fcca40b81c0 2023-07-06T21:42:20.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 -- 172.21.15.84:0/340120621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40b81c0 msgr2=0x7fcca40cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 --2- 172.21.15.84:0/340120621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40b81c0 0x7fcca40cf600 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fcca40c9c50 tx=0x7fcc94009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 -- 172.21.15.84:0/340120621 shutdown_connections 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 --2- 172.21.15.84:0/340120621 >> v2:172.21.15.107:3300/0 conn(0x7fcca40ceda0 0x7fcca40cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 --2- 172.21.15.84:0/340120621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40b81c0 0x7fcca40cf600 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 -- 172.21.15.84:0/340120621 >> 172.21.15.84:0/340120621 conn(0x7fcca406b2b0 msgr2=0x7fcca406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 -- 172.21.15.84:0/340120621 shutdown_connections 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.821+0000 7fcca9bff700 1 -- 172.21.15.84:0/340120621 wait complete. 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.823+0000 7fcca9bff700 1 Processor -- start 2023-07-06T21:42:20.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.823+0000 7fcca9bff700 1 -- start start 2023-07-06T21:42:20.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca9bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcca40b1a80 0x7fcca4142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca9bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcca40b81c0 0x7fcca41440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca9bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 0x7fcca4144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca9bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4142aa0 con 0x7fcca40ceda0 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca9bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4142d80 con 0x7fcca40b81c0 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca9bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4143060 con 0x7fcca40b1a80 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 0x7fcca4144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 0x7fcca4144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57470/0 (socket says 172.21.15.84:57470) 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 -- 172.21.15.84:0/2467595761 learned_addr learned my addr 172.21.15.84:0/2467595761 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca37fe700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcca40b1a80 0x7fcca4142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcc9bfff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcca40b81c0 0x7fcca41440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 -- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcca40b1a80 msgr2=0x7fcca4142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcca40b1a80 0x7fcca4142050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 -- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcca40b81c0 msgr2=0x7fcca41440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcca40b81c0 0x7fcca41440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca3fff700 1 -- 172.21.15.84:0/2467595761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc94009550 con 0x7fcca40ceda0 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcca37fe700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcca40b1a80 0x7fcca4142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:20.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.824+0000 7fcc9bfff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcca40b81c0 0x7fcca41440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.825+0000 7fcca3fff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 0x7fcca4144620 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00efc0 tx=0x7fcc9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.825+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00cd40 con 0x7fcca40ceda0 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.825+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c00f040 con 0x7fcca40ceda0 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.825+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca4143340 con 0x7fcca40ceda0 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.825+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0146c0 con 0x7fcca40ceda0 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.825+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca4144dc0 con 0x7fcca40ceda0 2023-07-06T21:42:20.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.826+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca4143860 con 0x7fcca40ceda0 2023-07-06T21:42:20.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.827+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fcc9c014820 con 0x7fcca40ceda0 2023-07-06T21:42:20.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.828+0000 7fcca0ff9700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcc8406ace0 0x7fcc8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:20.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.828+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcc9c089de0 con 0x7fcca40ceda0 2023-07-06T21:42:20.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.828+0000 7fcca37fe700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcc8406ace0 0x7fcc8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:20.833 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.829+0000 7fcca37fe700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcc8406ace0 0x7fcc8406d190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009fd0 tx=0x7fcc94008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:20.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:20.832+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fcca4143860 con 0x7fcca40ceda0 2023-07-06T21:42:21.029 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.027+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcca4143860 con 0x7fcca40ceda0 2023-07-06T21:42:21.030 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.027+0000 7fcca0ff9700 1 -- 172.21.15.84:0/2467595761 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc9c018020 con 0x7fcca40ceda0 2023-07-06T21:42:21.030 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcc8406ace0 msgr2=0x7fcc8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcc8406ace0 0x7fcc8406d190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcc94009fd0 tx=0x7fcc94008030 comp rx=0 tx=0).stop 2023-07-06T21:42:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 msgr2=0x7fcca4144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 0x7fcca4144620 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00efc0 tx=0x7fcc9c00c3a0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.032 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 shutdown_connections 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fcc8406ace0 0x7fcc8406d190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcca40ceda0 0x7fcca4144620 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcca40b1a80 0x7fcca4142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 --2- 172.21.15.84:0/2467595761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcca40b81c0 0x7fcca41440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 >> 172.21.15.84:0/2467595761 conn(0x7fcca406b2b0 msgr2=0x7fcca40d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 shutdown_connections 2023-07-06T21:42:21.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.030+0000 7fcca9bff700 1 -- 172.21.15.84:0/2467595761 wait complete. 2023-07-06T21:42:21.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:21.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.244+0000 7f77d71ff700 1 Processor -- start 2023-07-06T21:42:21.247 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.244+0000 7f77d71ff700 1 -- start start 2023-07-06T21:42:21.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d71ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b1a80 0x7f77d00d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d71ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00d1830 con 0x7f77d00b81c0 2023-07-06T21:42:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d71ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00cf330 con 0x7f77d00cfd10 2023-07-06T21:42:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d71ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d00d1270 con 0x7f77d00b1a80 2023-07-06T21:42:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d4f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f77d00b81c0 0x7f77d00cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56206/0 (socket says 172.21.15.84:56206) 2023-07-06T21:42:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d4f9b700 1 -- 172.21.15.84:0/2658548041 learned_addr learned my addr 172.21.15.84:0/2658548041 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77d579c700 1 --2- 172.21.15.84:0/2658548041 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b1a80 0x7f77d00d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133000129 0 0) 0x7f77d00d1830 con 0x7f77d00b81c0 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77c00035b0 con 0x7f77d00b81c0 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.246+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155589516 0 0) 0x7f77d00cf330 con 0x7f77d00cfd10 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77d00d1830 con 0x7f77d00cfd10 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77d579c700 1 -- 172.21.15.84:0/2658548041 >> v1:172.21.15.107:6789/0 conn(0x7f77d00cfd10 legacy=0x7f77d00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77d579c700 1 -- 172.21.15.84:0/2658548041 >> v1:172.21.15.84:6789/0 conn(0x7f77d00b81c0 legacy=0x7f77d00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2908033643 0 0) 0x7f77c00035b0 con 0x7f77d00b81c0 2023-07-06T21:42:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77d579c700 1 -- 172.21.15.84:0/2658548041 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00d1550 con 0x7f77d00b1a80 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77d579c700 1 --2- 172.21.15.84:0/2658548041 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b1a80 0x7f77d00d2a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009dc0 tx=0x7f77c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d56fc43fe3d1dff server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800e070 con 0x7f77d00b1a80 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8003a20 con 0x7f77d00b1a80 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.247+0000 7f77cf7fe700 1 -- 172.21.15.84:0/2658548041 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4001670 con 0x7f77d00b1a80 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 -- 172.21.15.84:0/2658548041 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b1a80 msgr2=0x7f77d00d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 --2- 172.21.15.84:0/2658548041 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b1a80 0x7f77d00d2a90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009dc0 tx=0x7f77c800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 -- 172.21.15.84:0/2658548041 shutdown_connections 2023-07-06T21:42:21.254 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 --2- 172.21.15.84:0/2658548041 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b1a80 0x7f77d00d2a90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 -- 172.21.15.84:0/2658548041 >> 172.21.15.84:0/2658548041 conn(0x7f77d006b2b0 msgr2=0x7f77d006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:21.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 -- 172.21.15.84:0/2658548041 shutdown_connections 2023-07-06T21:42:21.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.248+0000 7f77d71ff700 1 -- 172.21.15.84:0/2658548041 wait complete. 2023-07-06T21:42:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.250+0000 7f77d71ff700 1 Processor -- start 2023-07-06T21:42:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.250+0000 7f77d71ff700 1 -- start start 2023-07-06T21:42:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d71ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b1a80 0x7f77d01f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d71ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b81c0 0x7f77d00c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d71ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 0x7f77d0150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d71ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0150da0 con 0x7f77d00cfd10 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d71ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01f6980 con 0x7f77d00b1a80 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d71ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d01f60e0 con 0x7f77d00b81c0 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 0x7f77d0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 0x7f77d0150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57484/0 (socket says 172.21.15.84:57484) 2023-07-06T21:42:21.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 -- 172.21.15.84:0/1100166391 learned_addr learned my addr 172.21.15.84:0/1100166391 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77cffff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b81c0 0x7f77d00c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d4f9b700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b1a80 0x7f77d01f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 -- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b81c0 msgr2=0x7f77d00c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b81c0 0x7f77d00c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 -- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b1a80 msgr2=0x7f77d01f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b1a80 0x7f77d01f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.251+0000 7f77d579c700 1 -- 172.21.15.84:0/1100166391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8009a70 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.252+0000 7f77d579c700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 0x7f77d0150860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f77c800b370 tx=0x7f77c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.252+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8004540 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.252+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c80046a0 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.252+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d01f63c0 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.252+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c801d470 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.252+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d00c8500 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.253+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d00c8360 con 0x7f77d00cfd10 2023-07-06T21:42:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.254+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f77c801d5d0 con 0x7f77d00cfd10 2023-07-06T21:42:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.255+0000 7f77cd7fa700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f77b406acb0 0x7f77b406d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.255+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77c808b200 con 0x7f77d00cfd10 2023-07-06T21:42:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.255+0000 7f77d4f9b700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f77b406acb0 0x7f77b406d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.256+0000 7f77d4f9b700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f77b406acb0 0x7f77b406d160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f77c4001380 tx=0x7f77c4001590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:21.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.259+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f77d00c8360 con 0x7f77d00cfd10 2023-07-06T21:42:21.456 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.454+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f77d00c8360 con 0x7f77d00cfd10 2023-07-06T21:42:21.457 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.455+0000 7f77cd7fa700 1 -- 172.21.15.84:0/1100166391 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f77d00c8360 con 0x7f77d00cfd10 2023-07-06T21:42:21.457 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f77b406acb0 msgr2=0x7f77b406d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f77b406acb0 0x7f77b406d160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f77c4001380 tx=0x7f77c4001590 comp rx=0 tx=0).stop 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 msgr2=0x7f77d0150860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 0x7f77d0150860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f77c800b370 tx=0x7f77c8004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 shutdown_connections 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f77b406acb0 0x7f77b406d160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77d00cfd10 0x7f77d0150860 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77d00b1a80 0x7f77d01f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.457+0000 7f77d71ff700 1 --2- 172.21.15.84:0/1100166391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77d00b81c0 0x7f77d00c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.458+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 >> 172.21.15.84:0/1100166391 conn(0x7f77d006b2b0 msgr2=0x7f77d00d3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.458+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 shutdown_connections 2023-07-06T21:42:21.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.458+0000 7f77d71ff700 1 -- 172.21.15.84:0/1100166391 wait complete. 2023-07-06T21:42:21.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:42:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.671+0000 7f21715ff700 1 Processor -- start 2023-07-06T21:42:21.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.671+0000 7f21715ff700 1 -- start start 2023-07-06T21:42:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f21715ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f216c0cfd10 0x7f216c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f21715ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0d1830 con 0x7f216c0b81c0 2023-07-06T21:42:21.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f21715ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0cf330 con 0x7f216c0b1a80 2023-07-06T21:42:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f21715ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0d1270 con 0x7f216c0cfd10 2023-07-06T21:42:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f216affd700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f216c0b81c0 0x7f216c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56220/0 (socket says 172.21.15.84:56220) 2023-07-06T21:42:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f216affd700 1 -- 172.21.15.84:0/33451944 learned_addr learned my addr 172.21.15.84:0/33451944 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:21.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.672+0000 7f216a7fc700 1 --2- 172.21.15.84:0/33451944 >> v2:172.21.15.107:3300/0 conn(0x7f216c0cfd10 0x7f216c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f2169ffb700 1 -- 172.21.15.84:0/33451944 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1556157997 0 0) 0x7f216c0d1830 con 0x7f216c0b81c0 2023-07-06T21:42:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f2169ffb700 1 -- 172.21.15.84:0/33451944 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21540035b0 con 0x7f216c0b81c0 2023-07-06T21:42:21.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f216a7fc700 1 -- 172.21.15.84:0/33451944 >> v1:172.21.15.107:6789/0 conn(0x7f216c0b1a80 legacy=0x7f216c0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f216a7fc700 1 -- 172.21.15.84:0/33451944 >> v1:172.21.15.84:6789/0 conn(0x7f216c0b81c0 legacy=0x7f216c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f216a7fc700 1 -- 172.21.15.84:0/33451944 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0d1550 con 0x7f216c0cfd10 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f216a7fc700 1 --2- 172.21.15.84:0/33451944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0cfd10 0x7f216c0d2380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2160009840 tx=0x7f2160009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3d5bc739b5b8f1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f2169ffb700 1 -- 172.21.15.84:0/33451944 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011830 con 0x7f216c0cfd10 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f2169ffb700 1 -- 172.21.15.84:0/33451944 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2160011990 con 0x7f216c0cfd10 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.673+0000 7f2169ffb700 1 -- 172.21.15.84:0/33451944 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2160011d20 con 0x7f216c0cfd10 2023-07-06T21:42:21.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 -- 172.21.15.84:0/33451944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0cfd10 msgr2=0x7f216c0d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 --2- 172.21.15.84:0/33451944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0cfd10 0x7f216c0d2380 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2160009840 tx=0x7f2160009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 -- 172.21.15.84:0/33451944 shutdown_connections 2023-07-06T21:42:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 --2- 172.21.15.84:0/33451944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0cfd10 0x7f216c0d2380 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 -- 172.21.15.84:0/33451944 >> 172.21.15.84:0/33451944 conn(0x7f216c06b2b0 msgr2=0x7f216c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 -- 172.21.15.84:0/33451944 shutdown_connections 2023-07-06T21:42:21.678 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.674+0000 7f21715ff700 1 -- 172.21.15.84:0/33451944 wait complete. 2023-07-06T21:42:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.676+0000 7f21715ff700 1 Processor -- start 2023-07-06T21:42:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.676+0000 7f21715ff700 1 -- start start 2023-07-06T21:42:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f21715ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f216c0b1a80 0x7f216c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f21715ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0b81c0 0x7f216c1e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f21715ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 0x7f216c1e4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f21715ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0cad10 con 0x7f216c0cfd10 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f21715ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1e2f80 con 0x7f216c0b81c0 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f21715ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1e3260 con 0x7f216c0b1a80 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f216b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 0x7f216c1e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f216b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 0x7f216c1e4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57492/0 (socket says 172.21.15.84:57492) 2023-07-06T21:42:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f216b7fe700 1 -- 172.21.15.84:0/790525219 learned_addr learned my addr 172.21.15.84:0/790525219 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f216affd700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f216c0b1a80 0x7f216c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.681 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.677+0000 7f216a7fc700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0b81c0 0x7f216c1e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f216b7fe700 1 -- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f216c0b1a80 msgr2=0x7f216c14df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f216b7fe700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f216c0b1a80 0x7f216c14df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f216b7fe700 1 -- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0b81c0 msgr2=0x7f216c1e1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f216b7fe700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0b81c0 0x7f216c1e1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f216b7fe700 1 -- 172.21.15.84:0/790525219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600094a0 con 0x7f216c0cfd10 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f216b7fe700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 0x7f216c1e4160 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f216400cc50 tx=0x7f216400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164003c00 con 0x7f216c0cfd10 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164003d60 con 0x7f216c0cfd10 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c1e3540 con 0x7f216c0cfd10 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400aab0 con 0x7f216c0cfd10 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.678+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c14e450 con 0x7f216c0cfd10 2023-07-06T21:42:21.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.679+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c0cae80 con 0x7f216c0cfd10 2023-07-06T21:42:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.680+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f216400ac10 con 0x7f216c0cfd10 2023-07-06T21:42:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.681+0000 7f21537fe700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f215806ace0 0x7f215806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.681+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2164089420 con 0x7f216c0cfd10 2023-07-06T21:42:21.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.682+0000 7f216affd700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f215806ace0 0x7f215806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:21.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.682+0000 7f216affd700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f215806ace0 0x7f215806d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f215c007fd0 tx=0x7f215c007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:21.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.685+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f21640544c0 con 0x7f216c0cfd10 2023-07-06T21:42:21.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.884+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f216c04e1d0 con 0x7f216c0cfd10 2023-07-06T21:42:21.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.885+0000 7f21537fe700 1 -- 172.21.15.84:0/790525219 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2164057ae0 con 0x7f216c0cfd10 2023-07-06T21:42:21.887 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:42:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f215806ace0 msgr2=0x7f215806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f215806ace0 0x7f215806d190 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f215c007fd0 tx=0x7f215c007dc0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 msgr2=0x7f216c1e4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 0x7f216c1e4160 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f216400cc50 tx=0x7f216400cf60 comp rx=0 tx=0).stop 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 shutdown_connections 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f215806ace0 0x7f215806d190 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f216c0cfd10 0x7f216c1e4160 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f216c0b1a80 0x7f216c14df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 --2- 172.21.15.84:0/790525219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216c0b81c0 0x7f216c1e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 >> 172.21.15.84:0/790525219 conn(0x7f216c06b2b0 msgr2=0x7f216c0d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 shutdown_connections 2023-07-06T21:42:21.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:21.887+0000 7f21715ff700 1 -- 172.21.15.84:0/790525219 wait complete. 2023-07-06T21:42:21.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:42:22.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.102+0000 7f3065bff700 1 Processor -- start 2023-07-06T21:42:22.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.102+0000 7f3065bff700 1 -- start start 2023-07-06T21:42:22.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f3065bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f30600caf40 0x7f30600ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f3065bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30600cb3c0 con 0x7f3060070af0 2023-07-06T21:42:22.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f3065bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306006bf00 con 0x7f306015c420 2023-07-06T21:42:22.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f3065bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306006b9b0 con 0x7f30600caf40 2023-07-06T21:42:22.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f3057fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3060070af0 0x7f30600b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:56230/0 (socket says 172.21.15.84:56230) 2023-07-06T21:42:22.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f3057fff700 1 -- 172.21.15.84:0/522483202 learned_addr learned my addr 172.21.15.84:0/522483202 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976732915 0 0) 0x7f3048001110 con 0x7f3060070af0 2023-07-06T21:42:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f304c0035b0 con 0x7f3060070af0 2023-07-06T21:42:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.104+0000 7f305ffff700 1 --2- 172.21.15.84:0/522483202 >> v2:172.21.15.107:3300/0 conn(0x7f30600caf40 0x7f30600ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624664483 0 0) 0x7f304c0035b0 con 0x7f3060070af0 2023-07-06T21:42:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 >> v2:172.21.15.107:3300/0 conn(0x7f30600caf40 msgr2=0x7f30600ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:22.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 --2- 172.21.15.84:0/522483202 >> v2:172.21.15.107:3300/0 conn(0x7f30600caf40 0x7f30600ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 >> v1:172.21.15.107:6789/0 conn(0x7f306015c420 legacy=0x7f306006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306006cfc0 con 0x7f3060070af0 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f3048001800 con 0x7f3060070af0 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 >> v1:172.21.15.84:6789/0 conn(0x7f3060070af0 legacy=0x7f30600b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c003a50 0x7f304c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304c004850 0x7f304c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c005250 0x7f304c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306006b9b0 con 0x7f304c005250 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c00c190 con 0x7f304c003a50 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c00c350 con 0x7f304c004850 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305ffff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c005250 0x7f304c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305f7fe700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c003a50 0x7f304c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f3057fff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304c004850 0x7f304c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305ffff700 1 -- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304c004850 msgr2=0x7f304c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:22.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305ffff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304c004850 0x7f304c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305ffff700 1 -- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c003a50 msgr2=0x7f304c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305ffff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c003a50 0x7f304c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f305ffff700 1 -- 172.21.15.84:0/522483202 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0035b0 con 0x7f304c005250 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.105+0000 7f3057fff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304c004850 0x7f304c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f305ffff700 1 -- 172.21.15.84:0/522483202 reap_dead start 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f305ffff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c005250 0x7f304c00ba10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f304c005830 tx=0x7f304c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785180d07c7bb16e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:22.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058007e20 con 0x7f304c005250 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058003a20 con 0x7f304c005250 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f305effd700 1 -- 172.21.15.84:0/522483202 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058003dd0 con 0x7f304c005250 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f3065bff700 1 -- 172.21.15.84:0/522483202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c005250 msgr2=0x7f304c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.106+0000 7f3065bff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c005250 0x7f304c00ba10 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f304c005830 tx=0x7f304c005880 comp rx=0 tx=0).stop 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.107+0000 7f3065bff700 1 -- 172.21.15.84:0/522483202 shutdown_connections 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.107+0000 7f3065bff700 1 --2- 172.21.15.84:0/522483202 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f304c005250 0x7f304c00ba10 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.107+0000 7f3065bff700 1 -- 172.21.15.84:0/522483202 >> 172.21.15.84:0/522483202 conn(0x7f306006e540 msgr2=0x7f306006e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:22.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.107+0000 7f3065bff700 1 -- 172.21.15.84:0/522483202 shutdown_connections 2023-07-06T21:42:22.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.107+0000 7f3065bff700 1 -- 172.21.15.84:0/522483202 wait complete. 2023-07-06T21:42:22.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.108+0000 7f3065bff700 1 Processor -- start 2023-07-06T21:42:22.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.109+0000 7f3065bff700 1 -- start start 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.109+0000 7f3065bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c005250 0x7f30601e8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.109+0000 7f3065bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b60c0 0x7f30600c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f3065bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 0x7f30600b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f3065bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600ca260 con 0x7f30600c8d50 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f3065bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c91c0 con 0x7f304c005250 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f3065bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c94a0 con 0x7f30600b60c0 2023-07-06T21:42:22.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 0x7f30600b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 0x7f30600b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57516/0 (socket says 172.21.15.84:57516) 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 -- 172.21.15.84:0/1835288481 learned_addr learned my addr 172.21.15.84:0/1835288481 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f3057fff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b60c0 0x7f30600c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305f7fe700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c005250 0x7f30601e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 -- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b60c0 msgr2=0x7f30600c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b60c0 0x7f30600c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 -- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c005250 msgr2=0x7f30601e8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c005250 0x7f30601e8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 -- 172.21.15.84:0/1835288481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600c9780 con 0x7f30600c8d50 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.110+0000 7f305ffff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 0x7f30600b3be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f306006f990 tx=0x7f3058004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.111+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580043e0 con 0x7f30600c8d50 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.111+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058004560 con 0x7f30600c8d50 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.111+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058004910 con 0x7f30600c8d50 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.111+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30600c9a60 con 0x7f30600c8d50 2023-07-06T21:42:22.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.111+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30601e89b0 con 0x7f30600c8d50 2023-07-06T21:42:22.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.112+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306004e1d0 con 0x7f30600c8d50 2023-07-06T21:42:22.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.113+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f305801f470 con 0x7f30600c8d50 2023-07-06T21:42:22.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.114+0000 7f305cff9700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f304006ace0 0x7f304006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.114+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f305808adc0 con 0x7f30600c8d50 2023-07-06T21:42:22.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.114+0000 7f305f7fe700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f304006ace0 0x7f304006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.115+0000 7f305f7fe700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f304006ace0 0x7f304006d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30600c5160 tx=0x7f3050009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:22.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.118+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3058055f80 con 0x7f30600c8d50 2023-07-06T21:42:22.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.315+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3060160c40 con 0x7f30600c8d50 2023-07-06T21:42:22.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.315+0000 7f305cff9700 1 -- 172.21.15.84:0/1835288481 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f305805de60 con 0x7f30600c8d50 2023-07-06T21:42:22.318 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:42:22.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f304006ace0 msgr2=0x7f304006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f304006ace0 0x7f304006d190 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f30600c5160 tx=0x7f3050009040 comp rx=0 tx=0).stop 2023-07-06T21:42:22.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 msgr2=0x7f30600b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 0x7f30600b3be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f306006f990 tx=0x7f3058004b40 comp rx=0 tx=0).stop 2023-07-06T21:42:22.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 shutdown_connections 2023-07-06T21:42:22.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f304006ace0 0x7f304006d190 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30600c8d50 0x7f30600b3be0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f304c005250 0x7f30601e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 --2- 172.21.15.84:0/1835288481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30600b60c0 0x7f30600c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 >> 172.21.15.84:0/1835288481 conn(0x7f306006e540 msgr2=0x7f30600ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 shutdown_connections 2023-07-06T21:42:22.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.318+0000 7f3065bff700 1 -- 172.21.15.84:0/1835288481 wait complete. 2023-07-06T21:42:22.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:42:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.529+0000 7fbee75ff700 1 Processor -- start 2023-07-06T21:42:22.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.529+0000 7fbee75ff700 1 -- start start 2023-07-06T21:42:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee75ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbee0154910 0x7fbee0070c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee75ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee0070230 con 0x7fbee015a7e0 2023-07-06T21:42:22.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee75ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee015b2a0 con 0x7fbee0154910 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee015b580 con 0x7fbee00c9410 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee539b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbee0154910 0x7fbee0070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee539b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fbee0154910 0x7fbee0070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57528/0 (socket says 172.21.15.84:57528) 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.530+0000 7fbee539b700 1 -- 172.21.15.84:0/344222094 learned_addr learned my addr 172.21.15.84:0/344222094 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee5b9c700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee539b700 1 -- 172.21.15.84:0/344222094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 msgr2=0x7fbee015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:22.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee539b700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee015ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee539b700 1 -- 172.21.15.84:0/344222094 >> v1:172.21.15.107:6789/0 conn(0x7fbee015a7e0 legacy=0x7fbee00c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee539b700 1 -- 172.21.15.84:0/344222094 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0070480 con 0x7fbee0154910 2023-07-06T21:42:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee5b9c700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:22.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbee539b700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee0154910 0x7fbee0070c80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098f0 tx=0x7fbedc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667072cfa7430fcd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:22.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbed7fff700 1 -- 172.21.15.84:0/344222094 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00b8a0 con 0x7fbee0154910 2023-07-06T21:42:22.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbed7fff700 1 -- 172.21.15.84:0/344222094 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbedc00ba00 con 0x7fbee0154910 2023-07-06T21:42:22.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.531+0000 7fbed7fff700 1 -- 172.21.15.84:0/344222094 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00bd90 con 0x7fbee0154910 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 -- 172.21.15.84:0/344222094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee0154910 msgr2=0x7fbee0070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee0154910 0x7fbee0070c80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbedc0098f0 tx=0x7fbedc009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 -- 172.21.15.84:0/344222094 shutdown_connections 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 --2- 172.21.15.84:0/344222094 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee0154910 0x7fbee0070c80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 -- 172.21.15.84:0/344222094 >> 172.21.15.84:0/344222094 conn(0x7fbee00cf3d0 msgr2=0x7fbee00cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:22.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 -- 172.21.15.84:0/344222094 shutdown_connections 2023-07-06T21:42:22.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.532+0000 7fbee75ff700 1 -- 172.21.15.84:0/344222094 wait complete. 2023-07-06T21:42:22.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.534+0000 7fbee75ff700 1 Processor -- start 2023-07-06T21:42:22.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.534+0000 7fbee75ff700 1 -- start start 2023-07-06T21:42:22.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee75ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee75ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee0154910 0x7fbee006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee75ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 0x7fbee0145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee75ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00ae4e0 con 0x7fbee015a7e0 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee75ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00c69e0 con 0x7fbee0154910 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee75ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee00c6cc0 con 0x7fbee00c9410 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 0x7fbee0145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 0x7fbee0145dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57536/0 (socket says 172.21.15.84:57536) 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 -- 172.21.15.84:0/3098971630 learned_addr learned my addr 172.21.15.84:0/3098971630 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:22.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 -- 172.21.15.84:0/3098971630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 msgr2=0x7fbee00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 -- 172.21.15.84:0/3098971630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee0154910 msgr2=0x7fbee006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee0154910 0x7fbee006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.535+0000 7fbee5b9c700 1 -- 172.21.15.84:0/3098971630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc009550 con 0x7fbee015a7e0 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.536+0000 7fbee5b9c700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 0x7fbee0145dd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbed8007ef0 tx=0x7fbed800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.536+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800de60 con 0x7fbee015a7e0 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.536+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed800f040 con 0x7fbee015a7e0 2023-07-06T21:42:22.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.536+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee00c6fa0 con 0x7fbee015a7e0 2023-07-06T21:42:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.536+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003c60 con 0x7fbee015a7e0 2023-07-06T21:42:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.536+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee00c66d0 con 0x7fbee015a7e0 2023-07-06T21:42:22.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.537+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee004e240 con 0x7fbee015a7e0 2023-07-06T21:42:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.538+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbed8004030 con 0x7fbee015a7e0 2023-07-06T21:42:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.539+0000 7fbed5ffb700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbec406ace0 0x7fbec406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.539+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbed8089770 con 0x7fbee015a7e0 2023-07-06T21:42:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.539+0000 7fbee539b700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbec406ace0 0x7fbec406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:22.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.540+0000 7fbee539b700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbec406ace0 0x7fbec406d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbedc000c00 tx=0x7fbedc010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:22.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.543+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbed8058210 con 0x7fbee015a7e0 2023-07-06T21:42:22.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.754+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbee00c88b0 con 0x7fbee015a7e0 2023-07-06T21:42:22.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.755+0000 7fbed5ffb700 1 -- 172.21.15.84:0/3098971630 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbed8057da0 con 0x7fbee015a7e0 2023-07-06T21:42:22.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbec406ace0 msgr2=0x7fbec406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbec406ace0 0x7fbec406d190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbedc000c00 tx=0x7fbedc010000 comp rx=0 tx=0).stop 2023-07-06T21:42:22.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 msgr2=0x7fbee0145dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:22.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 0x7fbee0145dd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fbed8007ef0 tx=0x7fbed800d3b0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 shutdown_connections 2023-07-06T21:42:22.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbec406ace0 0x7fbec406d190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbee015a7e0 0x7fbee0145dd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbee00c9410 0x7fbee00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 --2- 172.21.15.84:0/3098971630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbee0154910 0x7fbee006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:22.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 >> 172.21.15.84:0/3098971630 conn(0x7fbee00cf3d0 msgr2=0x7fbee00b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:22.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 shutdown_connections 2023-07-06T21:42:22.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:22.757+0000 7fbee75ff700 1 -- 172.21.15.84:0/3098971630 wait complete. 2023-07-06T21:42:22.764 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:42:22.777 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:22.778 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:22.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:23.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.054+0000 7f3aa57ff700 1 Processor -- start 2023-07-06T21:42:23.056 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.054+0000 7f3aa57ff700 1 -- start start 2023-07-06T21:42:23.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.055+0000 7f3aa57ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3aa00b83c0 0x7f3aa00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.057 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.055+0000 7f3aa57ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa00d0a80 con 0x7f3aa00d3710 2023-07-06T21:42:23.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.055+0000 7f3aa57ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa00b8790 con 0x7f3aa00cfc60 2023-07-06T21:42:23.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.055+0000 7f3aa57ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa00b4e70 con 0x7f3aa00b83c0 2023-07-06T21:42:23.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.055+0000 7f3a97fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3aa00d3710 0x7f3aa00cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58630/0 (socket says 172.21.15.84:58630) 2023-07-06T21:42:23.058 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.055+0000 7f3a97fff700 1 -- 172.21.15.84:0/1425263360 learned_addr learned my addr 172.21.15.84:0/1425263360 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:23.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1425263360 >> v2:172.21.15.107:3300/0 conn(0x7f3aa00b83c0 0x7f3aa00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724267582 0 0) 0x7f3aa00d0a80 con 0x7f3aa00d3710 2023-07-06T21:42:23.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a8c0035b0 con 0x7f3aa00d3710 2023-07-06T21:42:23.061 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4053419135 0 0) 0x7f3a8c0035b0 con 0x7f3aa00d3710 2023-07-06T21:42:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 >> v2:172.21.15.107:3300/0 conn(0x7f3aa00b83c0 msgr2=0x7f3aa00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 --2- 172.21.15.84:0/1425263360 >> v2:172.21.15.107:3300/0 conn(0x7f3aa00b83c0 0x7f3aa00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 >> v1:172.21.15.107:6789/0 conn(0x7f3aa00cfc60 legacy=0x7f3aa00d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa00b5150 con 0x7f3aa00d3710 2023-07-06T21:42:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f3a88001670 con 0x7f3aa00d3710 2023-07-06T21:42:23.062 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 >> v1:172.21.15.84:6789/0 conn(0x7f3aa00d3710 legacy=0x7f3aa00cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8c003a00 0x7f3a8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c004850 0x7f3a8c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a8c005200 0x7f3a8c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa00b4e70 con 0x7f3a8c003a00 2023-07-06T21:42:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c00c110 con 0x7f3a8c004850 2023-07-06T21:42:23.063 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c00c2d0 con 0x7f3a8c005200 2023-07-06T21:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.056+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8c003a00 0x7f3a8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a97fff700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c004850 0x7f3a8c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9f7fe700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a8c005200 0x7f3a8c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9effd700 1 -- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a8c005200 msgr2=0x7f3a8c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a8c005200 0x7f3a8c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9effd700 1 -- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c004850 msgr2=0x7f3a8c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c004850 0x7f3a8c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9effd700 1 -- 172.21.15.84:0/1425263360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0035b0 con 0x7f3a8c003a00 2023-07-06T21:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9f7fe700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a8c005200 0x7f3a8c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9f7fe700 1 -- 172.21.15.84:0/1425263360 reap_dead start 2023-07-06T21:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8c003a00 0x7f3a8c003e10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009b60 tx=0x7f3a8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f247983eabb76d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000e6c0 con 0x7f3a8c003a00 2023-07-06T21:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9000e820 con 0x7f3a8c003a00 2023-07-06T21:42:23.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.057+0000 7f3a9e7fc700 1 -- 172.21.15.84:0/1425263360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000eb90 con 0x7f3a8c003a00 2023-07-06T21:42:23.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1425263360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8c003a00 msgr2=0x7f3a8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.067 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8c003a00 0x7f3a8c003e10 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3a90009b60 tx=0x7f3a8c005880 comp rx=0 tx=0).stop 2023-07-06T21:42:23.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1425263360 shutdown_connections 2023-07-06T21:42:23.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1425263360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a8c003a00 0x7f3a8c003e10 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1425263360 >> 172.21.15.84:0/1425263360 conn(0x7f3aa006b350 msgr2=0x7f3aa006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:23.068 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1425263360 shutdown_connections 2023-07-06T21:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.058+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1425263360 wait complete. 2023-07-06T21:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.060+0000 7f3aa57ff700 1 Processor -- start 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.061+0000 7f3aa57ff700 1 -- start start 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3aa57ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c003a00 0x7f3aa0154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3aa57ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3aa00d29d0 0x7f3aa00d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3aa57ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 0x7f3aa00cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3aa57ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa00d3380 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3aa57ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa0156e70 con 0x7f3a8c003a00 2023-07-06T21:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3aa57ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa0157150 con 0x7f3aa00d29d0 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3a9f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 0x7f3aa00cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3a9f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 0x7f3aa00cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40332/0 (socket says 172.21.15.84:40332) 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3a9f7fe700 1 -- 172.21.15.84:0/1743635831 learned_addr learned my addr 172.21.15.84:0/1743635831 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c003a00 0x7f3aa0154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3a97fff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3aa00d29d0 0x7f3aa00d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.062+0000 7f3a9f7fe700 1 -- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3aa00d29d0 msgr2=0x7f3aa00d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a9f7fe700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3aa00d29d0 0x7f3aa00d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a9f7fe700 1 -- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c003a00 msgr2=0x7f3aa0154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a9f7fe700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c003a00 0x7f3aa0154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a9f7fe700 1 -- 172.21.15.84:0/1743635831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0157430 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a9f7fe700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 0x7f3aa00cb920 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00d06c0 tx=0x7f3a9800eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003c10 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98003d90 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa0157710 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.074 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98005310 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.063+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa0154580 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.064+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa0157c30 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.065+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3a98005470 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.066+0000 7f3a977fe700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3a8006ace0 0x7f3a8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.066+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a98089330 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.066+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3a8006ace0 0x7f3a8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.067+0000 7f3a9effd700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3a8006ace0 0x7f3a8006d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3a900090b0 tx=0x7f3a9001e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.070+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f3aa0157c30 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.270+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3aa0157c30 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.271+0000 7f3a977fe700 1 -- 172.21.15.84:0/1743635831 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7f3aa0157c30 con 0x7f3aa00cb4b0 2023-07-06T21:42:23.274 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:23.274 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:23.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3a8006ace0 msgr2=0x7f3a8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3a8006ace0 0x7f3a8006d190 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3a900090b0 tx=0x7f3a9001e040 comp rx=0 tx=0).stop 2023-07-06T21:42:23.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 msgr2=0x7f3aa00cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 0x7f3aa00cb920 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00d06c0 tx=0x7f3a9800eda0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 shutdown_connections 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3a8006ace0 0x7f3a8006d190 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa00cb4b0 0x7f3aa00cb920 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a8c003a00 0x7f3aa0154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 --2- 172.21.15.84:0/1743635831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3aa00d29d0 0x7f3aa00d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 >> 172.21.15.84:0/1743635831 conn(0x7f3aa006b350 msgr2=0x7f3aa00732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:23.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 shutdown_connections 2023-07-06T21:42:23.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.274+0000 7f3aa57ff700 1 -- 172.21.15.84:0/1743635831 wait complete. 2023-07-06T21:42:23.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:42:08.774391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:42:09.558402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:23.295 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:23.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.489+0000 7fc5e71ff700 1 Processor -- start 2023-07-06T21:42:23.492 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.489+0000 7fc5e71ff700 1 -- start start 2023-07-06T21:42:23.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e71ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc5e00caf40 0x7fc5e00ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e71ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e00cb3c0 con 0x7fc5e015c420 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e71ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e006bf00 con 0x7fc5e0070af0 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e71ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e006b9b0 con 0x7fc5e00caf40 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e4f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc5e015c420 0x7fc5e006c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58646/0 (socket says 172.21.15.84:58646) 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e4f9b700 1 -- 172.21.15.84:0/2174520812 learned_addr learned my addr 172.21.15.84:0/2174520812 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.491+0000 7fc5e579c700 1 --2- 172.21.15.84:0/2174520812 >> v2:172.21.15.84:3300/0 conn(0x7fc5e00caf40 0x7fc5e00ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5df7fe700 1 -- 172.21.15.84:0/2174520812 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254505207 0 0) 0x7fc5d4001110 con 0x7fc5e015c420 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5df7fe700 1 -- 172.21.15.84:0/2174520812 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d00035b0 con 0x7fc5e015c420 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5e579c700 1 -- 172.21.15.84:0/2174520812 >> v1:172.21.15.107:6789/0 conn(0x7fc5e0070af0 legacy=0x7fc5e00b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5e579c700 1 -- 172.21.15.84:0/2174520812 >> v1:172.21.15.84:6789/0 conn(0x7fc5e015c420 legacy=0x7fc5e006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5e579c700 1 -- 172.21.15.84:0/2174520812 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e006cfc0 con 0x7fc5e00caf40 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5e579c700 1 --2- 172.21.15.84:0/2174520812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e00caf40 0x7fc5e00ce730 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009d90 tx=0x7fc5d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3845e23606f5c2a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.492+0000 7fc5df7fe700 1 -- 172.21.15.84:0/2174520812 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800e070 con 0x7fc5e00caf40 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5df7fe700 1 -- 172.21.15.84:0/2174520812 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d800bb20 con 0x7fc5e00caf40 2023-07-06T21:42:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5df7fe700 1 -- 172.21.15.84:0/2174520812 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8012430 con 0x7fc5e00caf40 2023-07-06T21:42:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 -- 172.21.15.84:0/2174520812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e00caf40 msgr2=0x7fc5e00ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/2174520812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e00caf40 0x7fc5e00ce730 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc5d8009d90 tx=0x7fc5d800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 -- 172.21.15.84:0/2174520812 shutdown_connections 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/2174520812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e00caf40 0x7fc5e00ce730 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 -- 172.21.15.84:0/2174520812 >> 172.21.15.84:0/2174520812 conn(0x7fc5e006e540 msgr2=0x7fc5e006e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 -- 172.21.15.84:0/2174520812 shutdown_connections 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.493+0000 7fc5e71ff700 1 -- 172.21.15.84:0/2174520812 wait complete. 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.495+0000 7fc5e71ff700 1 Processor -- start 2023-07-06T21:42:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.495+0000 7fc5e71ff700 1 -- start start 2023-07-06T21:42:23.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e71ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e0070af0 0x7fc5e015f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e71ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e00caf40 0x7fc5e00c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e71ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 0x7fc5e01575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.500 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e71ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015ee90 con 0x7fc5e015c420 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e71ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015f860 con 0x7fc5e00caf40 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e71ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e015e280 con 0x7fc5e0070af0 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e579c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 0x7fc5e01575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e579c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 0x7fc5e01575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40338/0 (socket says 172.21.15.84:40338) 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e579c700 1 -- 172.21.15.84:0/938419106 learned_addr learned my addr 172.21.15.84:0/938419106 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5dffff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e00caf40 0x7fc5e00c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.496+0000 7fc5e4f9b700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e0070af0 0x7fc5e015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e579c700 1 -- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e0070af0 msgr2=0x7fc5e015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e579c700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e0070af0 0x7fc5e015f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e579c700 1 -- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e00caf40 msgr2=0x7fc5e00c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e579c700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e00caf40 0x7fc5e00c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e579c700 1 -- 172.21.15.84:0/938419106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d8009a70 con 0x7fc5e015c420 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e4f9b700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e0070af0 0x7fc5e015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e579c700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 0x7fc5e01575f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00b34e0 tx=0x7fc5d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8012430 con 0x7fc5e015c420 2023-07-06T21:42:23.502 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d8020070 con 0x7fc5e015c420 2023-07-06T21:42:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8004280 con 0x7fc5e015c420 2023-07-06T21:42:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e015e560 con 0x7fc5e015c420 2023-07-06T21:42:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.497+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e015eb20 con 0x7fc5e015c420 2023-07-06T21:42:23.503 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.498+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e004e1d0 con 0x7fc5e015c420 2023-07-06T21:42:23.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.499+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc5d80043e0 con 0x7fc5e015c420 2023-07-06T21:42:23.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.500+0000 7fc5dd7fa700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc5c406ace0 0x7fc5c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.500+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc5d8022070 con 0x7fc5e015c420 2023-07-06T21:42:23.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.501+0000 7fc5e4f9b700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc5c406ace0 0x7fc5c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.504 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.501+0000 7fc5e4f9b700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc5c406ace0 0x7fc5c406d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4001270 tx=0x7fc5d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.506 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.504+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc5d80bd310 con 0x7fc5e015c420 2023-07-06T21:42:23.704 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.702+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5e006f420 con 0x7fc5e015c420 2023-07-06T21:42:23.705 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.703+0000 7fc5dd7fa700 1 -- 172.21.15.84:0/938419106 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5d802c020 con 0x7fc5e015c420 2023-07-06T21:42:23.705 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:23.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc5c406ace0 msgr2=0x7fc5c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc5c406ace0 0x7fc5c406d190 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4001270 tx=0x7fc5d4009040 comp rx=0 tx=0).stop 2023-07-06T21:42:23.707 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 msgr2=0x7fc5e01575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 0x7fc5e01575f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc5e00b34e0 tx=0x7fc5d8004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 shutdown_connections 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc5c406ace0 0x7fc5c406d190 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc5e015c420 0x7fc5e01575f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e0070af0 0x7fc5e015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 --2- 172.21.15.84:0/938419106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e00caf40 0x7fc5e00c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 >> 172.21.15.84:0/938419106 conn(0x7fc5e006e540 msgr2=0x7fc5e00b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 shutdown_connections 2023-07-06T21:42:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.705+0000 7fc5e71ff700 1 -- 172.21.15.84:0/938419106 wait complete. 2023-07-06T21:42:23.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:23.920 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.918+0000 7fa8645ff700 1 Processor -- start 2023-07-06T21:42:23.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.919+0000 7fa8645ff700 1 -- start start 2023-07-06T21:42:23.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa8645ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa85c0b81c0 0x7fa85c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa8645ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c0d1830 con 0x7fa85c0ceda0 2023-07-06T21:42:23.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa8645ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c160d40 con 0x7fa85c0b1a80 2023-07-06T21:42:23.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa8645ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85c0cf330 con 0x7fa85c0b81c0 2023-07-06T21:42:23.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa862b9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fa85c0ceda0 0x7fa85c160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58654/0 (socket says 172.21.15.84:58654) 2023-07-06T21:42:23.923 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa862b9c700 1 -- 172.21.15.84:0/3074830248 learned_addr learned my addr 172.21.15.84:0/3074830248 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa86239b700 1 --2- 172.21.15.84:0/3074830248 >> v2:172.21.15.107:3300/0 conn(0x7fa85c0b81c0 0x7fa85c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.920+0000 7fa861399700 1 -- 172.21.15.84:0/3074830248 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510298129 0 0) 0x7fa85c0d1830 con 0x7fa85c0ceda0 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa861399700 1 -- 172.21.15.84:0/3074830248 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8480035b0 con 0x7fa85c0ceda0 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa86239b700 1 -- 172.21.15.84:0/3074830248 >> v1:172.21.15.107:6789/0 conn(0x7fa85c0b1a80 legacy=0x7fa85c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa86239b700 1 -- 172.21.15.84:0/3074830248 >> v1:172.21.15.84:6789/0 conn(0x7fa85c0ceda0 legacy=0x7fa85c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa861399700 1 -- 172.21.15.84:0/3074830248 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319614873 0 0) 0x7fa8480035b0 con 0x7fa85c0ceda0 2023-07-06T21:42:23.924 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa86239b700 1 -- 172.21.15.84:0/3074830248 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0d1270 con 0x7fa85c0b81c0 2023-07-06T21:42:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa86239b700 1 --2- 172.21.15.84:0/3074830248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0b81c0 0x7fa85c0cf600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa858009840 tx=0x7fa858009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaede8ed26fa5e9f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa861399700 1 -- 172.21.15.84:0/3074830248 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540019f0 con 0x7fa85c0b81c0 2023-07-06T21:42:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa861399700 1 -- 172.21.15.84:0/3074830248 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858004030 con 0x7fa85c0b81c0 2023-07-06T21:42:23.925 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.921+0000 7fa861399700 1 -- 172.21.15.84:0/3074830248 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8580043c0 con 0x7fa85c0b81c0 2023-07-06T21:42:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 -- 172.21.15.84:0/3074830248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0b81c0 msgr2=0x7fa85c0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 --2- 172.21.15.84:0/3074830248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0b81c0 0x7fa85c0cf600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa858009840 tx=0x7fa858009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 -- 172.21.15.84:0/3074830248 shutdown_connections 2023-07-06T21:42:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 --2- 172.21.15.84:0/3074830248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0b81c0 0x7fa85c0cf600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 -- 172.21.15.84:0/3074830248 >> 172.21.15.84:0/3074830248 conn(0x7fa85c06b2b0 msgr2=0x7fa85c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:23.926 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 -- 172.21.15.84:0/3074830248 shutdown_connections 2023-07-06T21:42:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.922+0000 7fa8645ff700 1 -- 172.21.15.84:0/3074830248 wait complete. 2023-07-06T21:42:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.924+0000 7fa8645ff700 1 Processor -- start 2023-07-06T21:42:23.927 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.924+0000 7fa8645ff700 1 -- start start 2023-07-06T21:42:23.928 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa8645ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 0x7fa85c1f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa8645ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa85c0b81c0 0x7fa85c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa8645ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0ceda0 0x7fa85c0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa8645ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0caa30 con 0x7fa85c0b1a80 2023-07-06T21:42:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa8645ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1549e0 con 0x7fa85c0ceda0 2023-07-06T21:42:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa8645ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c1e6820 con 0x7fa85c0b81c0 2023-07-06T21:42:23.929 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 0x7fa85c1f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 0x7fa85c1f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40346/0 (socket says 172.21.15.84:40346) 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 -- 172.21.15.84:0/1983019431 learned_addr learned my addr 172.21.15.84:0/1983019431 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa862b9c700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0ceda0 0x7fa85c0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 -- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa85c0b81c0 msgr2=0x7fa85c0b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa861b9a700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa85c0b81c0 0x7fa85c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa85c0b81c0 0x7fa85c0b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 -- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0ceda0 msgr2=0x7fa85c0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:23.930 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0ceda0 0x7fa85c0b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa86239b700 1 -- 172.21.15.84:0/1983019431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580094a0 con 0x7fa85c0b1a80 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.925+0000 7fa862b9c700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0ceda0 0x7fa85c0b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.926+0000 7fa86239b700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 0x7fa85c1f2c70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa858009b90 tx=0x7fa858004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.926+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858011830 con 0x7fa85c0b1a80 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.926+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85801f070 con 0x7fa85c0b1a80 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.926+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858011bd0 con 0x7fa85c0b1a80 2023-07-06T21:42:23.931 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.926+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c1e7240 con 0x7fa85c0b1a80 2023-07-06T21:42:23.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.926+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c1e7800 con 0x7fa85c0b1a80 2023-07-06T21:42:23.932 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.927+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c04e1d0 con 0x7fa85c0b1a80 2023-07-06T21:42:23.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.928+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fa858011d30 con 0x7fa85c0b1a80 2023-07-06T21:42:23.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.929+0000 7fa852ffd700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa84006ace0 0x7fa84006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:23.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.929+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa858021070 con 0x7fa85c0b1a80 2023-07-06T21:42:23.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.929+0000 7fa861b9a700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa84006ace0 0x7fa84006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:23.933 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.930+0000 7fa861b9a700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa84006ace0 0x7fa84006d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa85c15fe50 tx=0x7fa84c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:23.935 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:23.933+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fa858055fd0 con 0x7fa85c0b1a80 2023-07-06T21:42:24.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.135+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa85c0b4f30 con 0x7fa85c0b1a80 2023-07-06T21:42:24.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.136+0000 7fa852ffd700 1 -- 172.21.15.84:0/1983019431 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa85802b0a0 con 0x7fa85c0b1a80 2023-07-06T21:42:24.139 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:24.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa84006ace0 msgr2=0x7fa84006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa84006ace0 0x7fa84006d190 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa85c15fe50 tx=0x7fa84c009040 comp rx=0 tx=0).stop 2023-07-06T21:42:24.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 msgr2=0x7fa85c1f2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 0x7fa85c1f2c70 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa858009b90 tx=0x7fa858004750 comp rx=0 tx=0).stop 2023-07-06T21:42:24.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 shutdown_connections 2023-07-06T21:42:24.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fa84006ace0 0x7fa84006d190 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa85c0ceda0 0x7fa85c0b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa85c0b1a80 0x7fa85c1f2c70 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 --2- 172.21.15.84:0/1983019431 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa85c0b81c0 0x7fa85c0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 >> 172.21.15.84:0/1983019431 conn(0x7fa85c06b2b0 msgr2=0x7fa85c15c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:24.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 shutdown_connections 2023-07-06T21:42:24.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.138+0000 7fa8645ff700 1 -- 172.21.15.84:0/1983019431 wait complete. 2023-07-06T21:42:24.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:42:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.353+0000 7fd4da9ff700 1 Processor -- start 2023-07-06T21:42:24.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.353+0000 7fd4da9ff700 1 -- start start 2023-07-06T21:42:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.354+0000 7fd4da9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40cfd10 0x7fd4d40d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.354+0000 7fd4da9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd4d40b1a80 0x7fd4d40d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.354+0000 7fd4da9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40d1830 con 0x7fd4d40b81c0 2023-07-06T21:42:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.354+0000 7fd4da9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40cf330 con 0x7fd4d40b1a80 2023-07-06T21:42:24.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.354+0000 7fd4da9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d40d1270 con 0x7fd4d40cfd10 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d3fff700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fd4d40b81c0 0x7fd4d40cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:54478/0 (socket says 172.21.15.84:54478) 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d37fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40cfd10 0x7fd4d40d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d8f9c700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd4d40b1a80 0x7fd4d40d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d3fff700 1 -- 172.21.15.84:0/1411192867 learned_addr learned my addr 172.21.15.84:0/1411192867 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d2ffd700 1 -- 172.21.15.84:0/1411192867 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223853121 0 0) 0x7fd4d40d1830 con 0x7fd4d40b81c0 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d2ffd700 1 -- 172.21.15.84:0/1411192867 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4bc0035b0 con 0x7fd4d40b81c0 2023-07-06T21:42:24.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d8f9c700 1 -- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40cfd10 msgr2=0x7fd4d40d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d8f9c700 1 --2- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40cfd10 0x7fd4d40d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d8f9c700 1 -- 172.21.15.84:0/1411192867 >> v1:172.21.15.107:6789/0 conn(0x7fd4d40b81c0 legacy=0x7fd4d40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d8f9c700 1 -- 172.21.15.84:0/1411192867 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40d1550 con 0x7fd4d40b1a80 2023-07-06T21:42:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.355+0000 7fd4d37fe700 1 --2- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40cfd10 0x7fd4d40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:24.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.356+0000 7fd4d8f9c700 1 --2- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40d28c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009dc0 tx=0x7fd4cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca3dfd222eadd29 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.356+0000 7fd4d2ffd700 1 -- 172.21.15.84:0/1411192867 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00e070 con 0x7fd4d40b1a80 2023-07-06T21:42:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.356+0000 7fd4d2ffd700 1 -- 172.21.15.84:0/1411192867 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00bae0 con 0x7fd4d40b1a80 2023-07-06T21:42:24.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.356+0000 7fd4d2ffd700 1 -- 172.21.15.84:0/1411192867 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00be90 con 0x7fd4d40b1a80 2023-07-06T21:42:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.356+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40b1a80 msgr2=0x7fd4d40d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.356+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40d28c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009dc0 tx=0x7fd4cc00b330 comp rx=0 tx=0).stop 2023-07-06T21:42:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.357+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1411192867 shutdown_connections 2023-07-06T21:42:24.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.357+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40d28c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.357+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1411192867 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40cfd10 0x7fd4d40d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.357+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1411192867 >> 172.21.15.84:0/1411192867 conn(0x7fd4d406b2b0 msgr2=0x7fd4d406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.357+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1411192867 shutdown_connections 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.357+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1411192867 wait complete. 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.358+0000 7fd4da9ff700 1 Processor -- start 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.358+0000 7fd4da9ff700 1 -- start start 2023-07-06T21:42:24.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.359+0000 7fd4da9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.359+0000 7fd4da9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40b81c0 0x7fd4d414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.359+0000 7fd4da9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40cfd10 0x7fd4d41f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.359+0000 7fd4da9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415de10 con 0x7fd4d40b1a80 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.359+0000 7fd4da9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415dae0 con 0x7fd4d40cfd10 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.359+0000 7fd4da9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d41f2740 con 0x7fd4d40b81c0 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40350/0 (socket says 172.21.15.84:40350) 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 -- 172.21.15.84:0/1416545538 learned_addr learned my addr 172.21.15.84:0/1416545538 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d37fe700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40b81c0 0x7fd4d414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d8f9c700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40cfd10 0x7fd4d41f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 -- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40b81c0 msgr2=0x7fd4d414fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40b81c0 0x7fd4d414fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 -- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40cfd10 msgr2=0x7fd4d41f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40cfd10 0x7fd4d41f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 -- 172.21.15.84:0/1416545538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c80094a0 con 0x7fd4d40b1a80 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.360+0000 7fd4d3fff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40cc5e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c700 tx=0x7fd4c400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.361+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4005d70 con 0x7fd4d40b1a80 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.361+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c40070d0 con 0x7fd4d40b1a80 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.361+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4007460 con 0x7fd4d40b1a80 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.361+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc009a70 con 0x7fd4d40b1a80 2023-07-06T21:42:24.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.361+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d41f2d00 con 0x7fd4d40b1a80 2023-07-06T21:42:24.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.362+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d404e1d0 con 0x7fd4d40b1a80 2023-07-06T21:42:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.363+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd4c40075c0 con 0x7fd4d40b1a80 2023-07-06T21:42:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.364+0000 7fd4d0ff9700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4c006ace0 0x7fd4c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.364+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd4c4088960 con 0x7fd4d40b1a80 2023-07-06T21:42:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.364+0000 7fd4d37fe700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4c006ace0 0x7fd4c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.364+0000 7fd4d37fe700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4c006ace0 0x7fd4c006d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b17a0 tx=0x7fd4c8009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:24.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.368+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd4c4053970 con 0x7fd4d40b1a80 2023-07-06T21:42:24.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.564+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4d40b4f30 con 0x7fd4d40b1a80 2023-07-06T21:42:24.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.564+0000 7fd4d0ff9700 1 -- 172.21.15.84:0/1416545538 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd4c4017020 con 0x7fd4d40b1a80 2023-07-06T21:42:24.567 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:42:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4c006ace0 msgr2=0x7fd4c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4c006ace0 0x7fd4c006d190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd4d40b17a0 tx=0x7fd4c8009210 comp rx=0 tx=0).stop 2023-07-06T21:42:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 msgr2=0x7fd4d40cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40cc5e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c700 tx=0x7fd4c400ca10 comp rx=0 tx=0).stop 2023-07-06T21:42:24.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 shutdown_connections 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd4c006ace0 0x7fd4c006d190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4d40cfd10 0x7fd4d41f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd4d40b1a80 0x7fd4d40cc5e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 --2- 172.21.15.84:0/1416545538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd4d40b81c0 0x7fd4d414fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 >> 172.21.15.84:0/1416545538 conn(0x7fd4d406b2b0 msgr2=0x7fd4d40b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 shutdown_connections 2023-07-06T21:42:24.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.567+0000 7fd4da9ff700 1 -- 172.21.15.84:0/1416545538 wait complete. 2023-07-06T21:42:24.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:42:24.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.782+0000 7f158fbff700 1 Processor -- start 2023-07-06T21:42:24.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.782+0000 7f158fbff700 1 -- start start 2023-07-06T21:42:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158fbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15880b81c0 0x7f15880cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f15880cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158fbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880d1830 con 0x7f15880ceda0 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158fbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880cf330 con 0x7f15880b81c0 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880d1270 con 0x7f15880b1a80 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15880b81c0 0x7f15880cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f15880b81c0 0x7f15880cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40352/0 (socket says 172.21.15.84:40352) 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 -- 172.21.15.84:0/3961892492 learned_addr learned my addr 172.21.15.84:0/3961892492 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d19a700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f15880cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 -- 172.21.15.84:0/3961892492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 msgr2=0x7f15880cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:24.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f15880cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 -- 172.21.15.84:0/3961892492 >> v1:172.21.15.107:6789/0 conn(0x7f15880ceda0 legacy=0x7f158815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d99b700 1 -- 172.21.15.84:0/3961892492 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880d1550 con 0x7f15880b81c0 2023-07-06T21:42:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.784+0000 7f158d19a700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f15880cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158d99b700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f15880cf600 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f15840098f0 tx=0x7f1584009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650f15946e68ea55 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158c999700 1 -- 172.21.15.84:0/3961892492 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400b8a0 con 0x7f15880b81c0 2023-07-06T21:42:24.789 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158c999700 1 -- 172.21.15.84:0/3961892492 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158400ba00 con 0x7f15880b81c0 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158c999700 1 -- 172.21.15.84:0/3961892492 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158400bd90 con 0x7f15880b81c0 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158fbff700 1 -- 172.21.15.84:0/3961892492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 msgr2=0x7f15880cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158fbff700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f15880cf600 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f15840098f0 tx=0x7f1584009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158fbff700 1 -- 172.21.15.84:0/3961892492 shutdown_connections 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158fbff700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f15880cf600 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158fbff700 1 --2- 172.21.15.84:0/3961892492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f15880cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.785+0000 7f158fbff700 1 -- 172.21.15.84:0/3961892492 >> 172.21.15.84:0/3961892492 conn(0x7f158806b2b0 msgr2=0x7f158806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:24.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.786+0000 7f158fbff700 1 -- 172.21.15.84:0/3961892492 shutdown_connections 2023-07-06T21:42:24.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.786+0000 7f158fbff700 1 -- 172.21.15.84:0/3961892492 wait complete. 2023-07-06T21:42:24.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.787+0000 7f158fbff700 1 Processor -- start 2023-07-06T21:42:24.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.787+0000 7f158fbff700 1 -- start start 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f158815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158fbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f158815a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158fbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880ceda0 0x7f15881e1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158fbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880c9ab0 con 0x7f15880b81c0 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158fbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880c7920 con 0x7f15880ceda0 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158815acc0 con 0x7f15880b1a80 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158d19a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f158815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.793 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158d19a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f158815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40368/0 (socket says 172.21.15.84:40368) 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.788+0000 7f158d19a700 1 -- 172.21.15.84:0/3092122477 learned_addr learned my addr 172.21.15.84:0/3092122477 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d99b700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f158815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158e19c700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880ceda0 0x7f15881e1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d19a700 1 -- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 msgr2=0x7f158815a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d19a700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f158815a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d19a700 1 -- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880ceda0 msgr2=0x7f15881e1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:24.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d19a700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880ceda0 0x7f15881e1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d19a700 1 -- 172.21.15.84:0/3092122477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584009550 con 0x7f15880b81c0 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158e19c700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880ceda0 0x7f15881e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d99b700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f158815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158d19a700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f158815a780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f157800e840 tx=0x7f157800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578013610 con 0x7f15880b81c0 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1578013770 con 0x7f15880b81c0 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1578013b20 con 0x7f15880b81c0 2023-07-06T21:42:24.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.789+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15880c97d0 con 0x7f15880b81c0 2023-07-06T21:42:24.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.790+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15880c75b0 con 0x7f15880b81c0 2023-07-06T21:42:24.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.791+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880c7ad0 con 0x7f15880b81c0 2023-07-06T21:42:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.792+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f1578013c80 con 0x7f15880b81c0 2023-07-06T21:42:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.793+0000 7f157e7fc700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f156c06ace0 0x7f156c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.793+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f15780890a0 con 0x7f15880b81c0 2023-07-06T21:42:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.793+0000 7f158d99b700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f156c06ace0 0x7f156c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:24.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.793+0000 7f158d99b700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f156c06ace0 0x7f156c06d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1584009c40 tx=0x7f1584008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:24.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.797+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f15880c7ad0 con 0x7f15880b81c0 2023-07-06T21:42:24.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.997+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15880c7ad0 con 0x7f15880b81c0 2023-07-06T21:42:25.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:24.999+0000 7f157e7fc700 1 -- 172.21.15.84:0/3092122477 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15880c7ad0 con 0x7f15880b81c0 2023-07-06T21:42:25.002 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:42:25.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f156c06ace0 msgr2=0x7f156c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f156c06ace0 0x7f156c06d190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1584009c40 tx=0x7f1584008030 comp rx=0 tx=0).stop 2023-07-06T21:42:25.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 msgr2=0x7f158815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f158815a780 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f157800e840 tx=0x7f157800ec00 comp rx=0 tx=0).stop 2023-07-06T21:42:25.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 shutdown_connections 2023-07-06T21:42:25.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f156c06ace0 0x7f156c06d190 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880ceda0 0x7f15881e1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b1a80 0x7f158815a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 --2- 172.21.15.84:0/3092122477 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15880b81c0 0x7f158815a780 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 >> 172.21.15.84:0/3092122477 conn(0x7f158806b2b0 msgr2=0x7f15880cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:25.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 shutdown_connections 2023-07-06T21:42:25.008 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.002+0000 7f158fbff700 1 -- 172.21.15.84:0/3092122477 wait complete. 2023-07-06T21:42:25.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:42:25.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.214+0000 7f45921ff700 1 Processor -- start 2023-07-06T21:42:25.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.214+0000 7f45921ff700 1 -- start start 2023-07-06T21:42:25.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.215+0000 7f45921ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f458c15c690 0x7f458c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f45921ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0d1860 con 0x7f458c0b4680 2023-07-06T21:42:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f45921ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f458c0cf4c0 con 0x7f458c15c2c0 2023-07-06T21:42:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f45921ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f458c0d1580 con 0x7f458c15c690 2023-07-06T21:42:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f4583fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f458c15c690 0x7f458c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f458bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f458c0b4680 0x7f458c0b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58668/0 (socket says 172.21.15.84:58668) 2023-07-06T21:42:25.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f458bfff700 1 -- 172.21.15.84:0/628096675 learned_addr learned my addr 172.21.15.84:0/628096675 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:25.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f4583fff700 1 -- 172.21.15.84:0/628096675 >> v1:172.21.15.107:6789/0 conn(0x7f458c15c2c0 legacy=0x7f458c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f4583fff700 1 -- 172.21.15.84:0/628096675 >> v1:172.21.15.84:6789/0 conn(0x7f458c0b4680 legacy=0x7f458c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f4583fff700 1 -- 172.21.15.84:0/628096675 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458c0cd380 con 0x7f458c15c690 2023-07-06T21:42:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f458affd700 1 -- 172.21.15.84:0/628096675 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138557049 0 0) 0x7f458c0d1860 con 0x7f458c0b4680 2023-07-06T21:42:25.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.216+0000 7f4583fff700 1 --2- 172.21.15.84:0/628096675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c15c690 0x7f458c0b5a50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f458c0cda10 tx=0x7f4574009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c571634a18996fc server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f458affd700 1 -- 172.21.15.84:0/628096675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574004020 con 0x7f458c15c690 2023-07-06T21:42:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f458affd700 1 -- 172.21.15.84:0/628096675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4574004180 con 0x7f458c15c690 2023-07-06T21:42:25.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f458affd700 1 -- 172.21.15.84:0/628096675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574004530 con 0x7f458c15c690 2023-07-06T21:42:25.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 -- 172.21.15.84:0/628096675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c15c690 msgr2=0x7f458c0b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 --2- 172.21.15.84:0/628096675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c15c690 0x7f458c0b5a50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f458c0cda10 tx=0x7f4574009b20 comp rx=0 tx=0).stop 2023-07-06T21:42:25.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 -- 172.21.15.84:0/628096675 shutdown_connections 2023-07-06T21:42:25.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 --2- 172.21.15.84:0/628096675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c15c690 0x7f458c0b5a50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 -- 172.21.15.84:0/628096675 >> 172.21.15.84:0/628096675 conn(0x7f458c0cf680 msgr2=0x7f458c0cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 -- 172.21.15.84:0/628096675 shutdown_connections 2023-07-06T21:42:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.217+0000 7f45921ff700 1 -- 172.21.15.84:0/628096675 wait complete. 2023-07-06T21:42:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.219+0000 7f45921ff700 1 Processor -- start 2023-07-06T21:42:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.219+0000 7f45921ff700 1 -- start start 2023-07-06T21:42:25.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f45921ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 0x7f458c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f45921ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f458c15c2c0 0x7f458c0b3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f45921ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f458c15c690 0x7f458c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f45921ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c15caf0 con 0x7f458c0b4680 2023-07-06T21:42:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f45921ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0cc4c0 con 0x7f458c15c2c0 2023-07-06T21:42:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f45921ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458c0cc7a0 con 0x7f458c15c690 2023-07-06T21:42:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f458b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 0x7f458c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f458b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 0x7f458c156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40392/0 (socket says 172.21.15.84:40392) 2023-07-06T21:42:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.220+0000 7f458b7fe700 1 -- 172.21.15.84:0/3095694340 learned_addr learned my addr 172.21.15.84:0/3095694340 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:25.230 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f4583fff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f458c15c2c0 0x7f458c0b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458bfff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f458c15c690 0x7f458c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458b7fe700 1 -- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f458c15c690 msgr2=0x7f458c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.231 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458b7fe700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f458c15c690 0x7f458c0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458b7fe700 1 -- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f458c15c2c0 msgr2=0x7f458c0b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458b7fe700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f458c15c2c0 0x7f458c0b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458b7fe700 1 -- 172.21.15.84:0/3095694340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45740094a0 con 0x7f458c0b4680 2023-07-06T21:42:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458bfff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f458c15c690 0x7f458c0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:25.232 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f458b7fe700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 0x7f458c156720 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f457c00c4e0 tx=0x7f457c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0052d0 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457c005430 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c0038c0 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458c0cca80 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.221+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c0cb790 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.222+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f458c0ce670 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.223+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f457c003a20 con 0x7f458c0b4680 2023-07-06T21:42:25.233 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.225+0000 7f4588ff9700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f456c06ace0 0x7f456c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.225+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f457c0889b0 con 0x7f458c0b4680 2023-07-06T21:42:25.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.225+0000 7f4583fff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f456c06ace0 0x7f456c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.225+0000 7f4583fff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f456c06ace0 0x7f456c06d190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f458c0cda10 tx=0x7f4574008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:25.234 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.229+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f457c053a50 con 0x7f458c0b4680 2023-07-06T21:42:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.436+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f458c04e240 con 0x7f458c0b4680 2023-07-06T21:42:25.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.436+0000 7f4588ff9700 1 -- 172.21.15.84:0/3095694340 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f457c057070 con 0x7f458c0b4680 2023-07-06T21:42:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f456c06ace0 msgr2=0x7f456c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f456c06ace0 0x7f456c06d190 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f458c0cda10 tx=0x7f4574008030 comp rx=0 tx=0).stop 2023-07-06T21:42:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 msgr2=0x7f458c156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 0x7f458c156720 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f457c00c4e0 tx=0x7f457c00c8a0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 shutdown_connections 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f456c06ace0 0x7f456c06d190 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f458c15c690 0x7f458c0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f458c0b4680 0x7f458c156720 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 --2- 172.21.15.84:0/3095694340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f458c15c2c0 0x7f458c0b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 >> 172.21.15.84:0/3095694340 conn(0x7f458c0cf680 msgr2=0x7f458c0d2400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 shutdown_connections 2023-07-06T21:42:25.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.439+0000 7f45921ff700 1 -- 172.21.15.84:0/3095694340 wait complete. 2023-07-06T21:42:25.444 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:42:25.460 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:42:25.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:25.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:25.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[153719]: fuse finished with error 0 and tester_r 0 2023-07-06T21:42:25.527 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:42:25.568 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:42:25.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:25.569 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:42:25.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:42:25.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.843+0000 7f8c0a3ff700 1 Processor -- start 2023-07-06T21:42:25.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.843+0000 7f8c0a3ff700 1 -- start start 2023-07-06T21:42:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c0a3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c040d2690 0x7f8c040cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c0a3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c040b59c0 0x7f8c0415fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c0a3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040d1460 con 0x7f8c040b1fd0 2023-07-06T21:42:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c0a3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040d2530 con 0x7f8c040d2690 2023-07-06T21:42:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c0a3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040d1740 con 0x7f8c040b59c0 2023-07-06T21:42:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c037fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c040d2690 0x7f8c040cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c037fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f8c040d2690 0x7f8c040cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40402/0 (socket says 172.21.15.84:40402) 2023-07-06T21:42:25.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.844+0000 7f8c037fe700 1 -- 172.21.15.84:0/3967071367 learned_addr learned my addr 172.21.15.84:0/3967071367 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c0899c700 1 --2- 172.21.15.84:0/3967071367 >> v2:172.21.15.107:3300/0 conn(0x7f8c040b59c0 0x7f8c0415fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c037fe700 1 -- 172.21.15.84:0/3967071367 >> v2:172.21.15.107:3300/0 conn(0x7f8c040b59c0 msgr2=0x7f8c0415fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c037fe700 1 --2- 172.21.15.84:0/3967071367 >> v2:172.21.15.107:3300/0 conn(0x7f8c040b59c0 0x7f8c0415fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c037fe700 1 -- 172.21.15.84:0/3967071367 >> v1:172.21.15.107:6789/0 conn(0x7f8c040b1fd0 legacy=0x7f8c040cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c037fe700 1 -- 172.21.15.84:0/3967071367 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040cd260 con 0x7f8c040d2690 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c037fe700 1 --2- 172.21.15.84:0/3967071367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040d2690 0x7f8c040cf870 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b8cb0 tx=0x7f8bf8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701477075ed4f9e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:25.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c02ffd700 1 -- 172.21.15.84:0/3967071367 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8011830 con 0x7f8c040d2690 2023-07-06T21:42:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c02ffd700 1 -- 172.21.15.84:0/3967071367 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf8011990 con 0x7f8c040d2690 2023-07-06T21:42:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.845+0000 7f8c02ffd700 1 -- 172.21.15.84:0/3967071367 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8011d20 con 0x7f8c040d2690 2023-07-06T21:42:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/3967071367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040d2690 msgr2=0x7f8c040cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/3967071367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040d2690 0x7f8c040cf870 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b8cb0 tx=0x7f8bf8009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/3967071367 shutdown_connections 2023-07-06T21:42:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/3967071367 >> v2:172.21.15.107:3300/0 conn(0x7f8c040b59c0 0x7f8c0415fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/3967071367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040d2690 0x7f8c040cf870 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/3967071367 >> 172.21.15.84:0/3967071367 conn(0x7f8c0406b340 msgr2=0x7f8c0406b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/3967071367 shutdown_connections 2023-07-06T21:42:25.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.846+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/3967071367 wait complete. 2023-07-06T21:42:25.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.848+0000 7f8c0a3ff700 1 Processor -- start 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.848+0000 7f8c0a3ff700 1 -- start start 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0a3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 0x7f8c041562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0a3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c040b59c0 0x7f8c041589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0a3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c040d2690 0x7f8c04158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0a3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04156d50 con 0x7f8c040b1fd0 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0a3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04156ff0 con 0x7f8c040d2690 2023-07-06T21:42:25.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0a3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c041572d0 con 0x7f8c040b59c0 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c03fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 0x7f8c041562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c03fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 0x7f8c041562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40414/0 (socket says 172.21.15.84:40414) 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c03fff700 1 -- 172.21.15.84:0/211840828 learned_addr learned my addr 172.21.15.84:0/211840828 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c037fe700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c040b59c0 0x7f8c041589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.849+0000 7f8c0899c700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c040d2690 0x7f8c04158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c03fff700 1 -- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c040b59c0 msgr2=0x7f8c041589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c03fff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c040b59c0 0x7f8c041589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c03fff700 1 -- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c040d2690 msgr2=0x7f8c04158f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c03fff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c040d2690 0x7f8c04158f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c03fff700 1 -- 172.21.15.84:0/211840828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf8009550 con 0x7f8c040b1fd0 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c0899c700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c040d2690 0x7f8c04158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c03fff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 0x7f8c041562f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c450 tx=0x7f8bf400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40052d0 con 0x7f8c040b1fd0 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4005430 con 0x7f8c040b1fd0 2023-07-06T21:42:25.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c041575b0 con 0x7f8c040b1fd0 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40038c0 con 0x7f8c040b1fd0 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.850+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c04156830 con 0x7f8c040b1fd0 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.851+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c041569c0 con 0x7f8c040b1fd0 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.852+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f8bf4003a20 con 0x7f8c040b1fd0 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.853+0000 7f8c00ff9700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8bf006ace0 0x7f8bf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.853+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8bf40888f0 con 0x7f8c040b1fd0 2023-07-06T21:42:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.853+0000 7f8c037fe700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8bf006ace0 0x7f8bf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.854+0000 7f8c037fe700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8bf006ace0 0x7f8bf006d190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b8cb0 tx=0x7f8bf8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:25.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:25.857+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8bf4053900 con 0x7f8c040b1fd0 2023-07-06T21:42:26.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.100+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8c0404e240 con 0x7f8c040b1fd0 2023-07-06T21:42:26.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.101+0000 7f8c00ff9700 1 -- 172.21.15.84:0/211840828 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8bf4017060 con 0x7f8c040b1fd0 2023-07-06T21:42:26.104 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:42:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.103+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8bf006ace0 msgr2=0x7f8bf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.103+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8bf006ace0 0x7f8bf006d190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c040b8cb0 tx=0x7f8bf8008030 comp rx=0 tx=0).stop 2023-07-06T21:42:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.103+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 msgr2=0x7f8c041562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.103+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 0x7f8c041562f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400c450 tx=0x7f8bf400c760 comp rx=0 tx=0).stop 2023-07-06T21:42:26.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 shutdown_connections 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f8bf006ace0 0x7f8bf006d190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c040d2690 0x7f8c04158f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8c040b1fd0 0x7f8c041562f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 --2- 172.21.15.84:0/211840828 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c040b59c0 0x7f8c041589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 >> 172.21.15.84:0/211840828 conn(0x7f8c0406b340 msgr2=0x7f8c040d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 shutdown_connections 2023-07-06T21:42:26.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:26.104+0000 7f8c0a3ff700 1 -- 172.21.15.84:0/211840828 wait complete. 2023-07-06T21:42:26.125 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:42:26.126 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:42:26.126 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:42:26.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:42:26.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:26.127 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:42:26.127 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:42:26.127 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:42:26.127 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:26.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.128 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:42:26.172 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:26.173 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:42:26.174 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42056sec preferred_lft 42056sec 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:42:26.175 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.176 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:42:26.177 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:42:26.177 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4492:bbff:fe16:eebb/64 scope link 2023-07-06T21:42:26.177 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:26.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.178 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:42:26.278 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:42:26.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.282 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:42:26.327 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:42:26.327 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:42:26.327 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:42:26.327 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:42:26.328 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:42:26.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.329 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:42:26.330 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:42:26.330 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:42:26.330 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:42:26.330 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:42:26.331 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:42:26.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.536 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:42:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:42:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:42:26.580 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:26.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:42:26.688 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:42:26.688 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:42:26.689 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:26.731 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:26.732 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:42:26.733 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:26.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:42:26.922 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:42:26.922 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:42:26.987 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:42:26.990 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:42:26.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:26.991 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:42:27.034 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:42:27.035 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:42:27.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:42:27.147 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:42:27.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:42:27.220 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:42:27.223 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:42:27.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:27.224 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:42:27.254 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:42:27.251+0000 7fd02931e5c0 -1 init, newargv = 0x564b360bff30 newargc=15 2023-07-06T21:42:27.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[154913]: starting ceph client 2023-07-06T21:42:27.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[154913]: starting fuse 2023-07-06T21:42:27.271 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:42:27.272 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:42:27.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:42:27.273 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:42:27.274 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:42:27.275 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:42:27.276 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:42:27.276 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:27.277 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:42:27.278 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:42:27.279 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:42:27.279 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:42:27.280 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:27.281 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:42:27.281 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:42:27.282 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:42:27.282 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:42:27.283 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:27.284 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:42:27.284 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:42:27.285 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:42:27.285 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:42:27.285 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:42:27.286 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:42:27.286 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:42:27.286 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:42:27.287 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:27.288 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:42:27.288 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:42:27.453 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.154913.asok 2023-07-06T21:42:27.630 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.154913.asok 2023-07-06T21:42:27.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:27.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.154913.asok status 2023-07-06T21:42:27.852 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:42:27.852 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:42:27.853 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:42:27.853 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:42:27.853 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:42:27.853 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:42:27.853 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:42:27.853 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "154913", 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6336, 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:42:27.854 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 6336 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3351290969 2023-07-06T21:42:27.855 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3351290969 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:27.856 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.6336 192.168.0.1:0/3351290969", 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/3351290969", 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 24, 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 53, 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:42:27.857 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:42:27.858 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:42:27.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:27.867 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:42:27.868 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:27.912 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:42:27.913 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:27.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:27.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:28.020 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:42:28.021 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:42:28.021 DEBUG:teuthology.orchestra.run.smithi084:> import errno 2023-07-06T21:42:28.021 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi084:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-07-06T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi084:> try: 2023-07-06T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi084:> if False: 2023-07-06T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi084:> ret = os.read(fd, 1024) 2023-07-06T21:42:28.023 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-07-06T21:42:28.023 DEBUG:teuthology.orchestra.run.smithi084:> os.write(fd, b'"'"'content'"'"') 2023-07-06T21:42:28.023 DEBUG:teuthology.orchestra.run.smithi084:> except OSError as e: 2023-07-06T21:42:28.023 DEBUG:teuthology.orchestra.run.smithi084:> if e.errno != errno.EPERM: 2023-07-06T21:42:28.024 DEBUG:teuthology.orchestra.run.smithi084:> raise 2023-07-06T21:42:28.024 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-07-06T21:42:28.024 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("client does not check permission of data pool") 2023-07-06T21:42:28.024 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2023-07-06T21:42:28.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.554+0000 7fe2445ff700 1 Processor -- start 2023-07-06T21:42:28.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.555+0000 7fe2445ff700 1 -- start start 2023-07-06T21:42:28.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe2445ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe23c0d0ea0 0x7fe23c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:28.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe2445ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 0x7fe23c160320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:28.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe2445ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe23c0ceef0 con 0x7fe23c160ac0 2023-07-06T21:42:28.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe2445ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe23c0cf450 con 0x7fe23c0d0ea0 2023-07-06T21:42:28.560 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe2445ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c160860 con 0x7fe23c15fee0 2023-07-06T21:42:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe242b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 0x7fe23c160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe242b9c700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 0x7fe23c160320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:57828/0 (socket says 172.21.15.84:57828) 2023-07-06T21:42:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe242b9c700 1 -- 172.21.15.84:0/589375009 learned_addr learned my addr 172.21.15.84:0/589375009 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.556+0000 7fe24239b700 1 --2- 172.21.15.84:0/589375009 >> v2:172.21.15.107:3300/0 conn(0x7fe23c0d0ea0 0x7fe23c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:28.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe242b9c700 1 -- 172.21.15.84:0/589375009 >> v2:172.21.15.107:3300/0 conn(0x7fe23c0d0ea0 msgr2=0x7fe23c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe242b9c700 1 --2- 172.21.15.84:0/589375009 >> v2:172.21.15.107:3300/0 conn(0x7fe23c0d0ea0 0x7fe23c0d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe242b9c700 1 -- 172.21.15.84:0/589375009 >> v1:172.21.15.107:6789/0 conn(0x7fe23c160ac0 legacy=0x7fe23c0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe242b9c700 1 -- 172.21.15.84:0/589375009 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0cf120 con 0x7fe23c15fee0 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe242b9c700 1 --2- 172.21.15.84:0/589375009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 0x7fe23c160320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe234009e50 tx=0x7fe23400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1725fb1920ae523 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe241399700 1 -- 172.21.15.84:0/589375009 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400e070 con 0x7fe23c15fee0 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe241399700 1 -- 172.21.15.84:0/589375009 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23400b910 con 0x7fe23c15fee0 2023-07-06T21:42:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.557+0000 7fe241399700 1 -- 172.21.15.84:0/589375009 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400bcc0 con 0x7fe23c15fee0 2023-07-06T21:42:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 -- 172.21.15.84:0/589375009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 msgr2=0x7fe23c160320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 --2- 172.21.15.84:0/589375009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 0x7fe23c160320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe234009e50 tx=0x7fe23400b330 comp rx=0 tx=0).stop 2023-07-06T21:42:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 -- 172.21.15.84:0/589375009 shutdown_connections 2023-07-06T21:42:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 --2- 172.21.15.84:0/589375009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c15fee0 0x7fe23c160320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 --2- 172.21.15.84:0/589375009 >> v2:172.21.15.107:3300/0 conn(0x7fe23c0d0ea0 0x7fe23c0d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 -- 172.21.15.84:0/589375009 >> 172.21.15.84:0/589375009 conn(0x7fe23c0715c0 msgr2=0x7fe23c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 -- 172.21.15.84:0/589375009 shutdown_connections 2023-07-06T21:42:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.558+0000 7fe2445ff700 1 -- 172.21.15.84:0/589375009 wait complete. 2023-07-06T21:42:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.560+0000 7fe2445ff700 1 Processor -- start 2023-07-06T21:42:28.566 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.561+0000 7fe2445ff700 1 -- start start 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe2445ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23c15fee0 0x7fe23c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe2445ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c160ac0 0x7fe23c0c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe2445ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe23c154fa0 0x7fe23c1f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe2445ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c0c6d10 con 0x7fe23c154fa0 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe2445ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c1f1ce0 con 0x7fe23c15fee0 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe2445ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c1f1fc0 con 0x7fe23c160ac0 2023-07-06T21:42:28.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe242b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe23c154fa0 0x7fe23c1f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:28.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe24239b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23c15fee0 0x7fe23c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:28.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe24239b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23c15fee0 0x7fe23c0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.84:40454/0 (socket says 172.21.15.84:40454) 2023-07-06T21:42:28.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe241b9a700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c160ac0 0x7fe23c0c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:28.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.562+0000 7fe24239b700 1 -- 172.21.15.84:0/2223748686 learned_addr learned my addr 172.21.15.84:0/2223748686 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:28.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe242b9c700 1 -- 172.21.15.84:0/2223748686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c160ac0 msgr2=0x7fe23c0c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:28.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe242b9c700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c160ac0 0x7fe23c0c67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe242b9c700 1 -- 172.21.15.84:0/2223748686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23c15fee0 msgr2=0x7fe23c0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:28.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe242b9c700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23c15fee0 0x7fe23c0c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe242b9c700 1 -- 172.21.15.84:0/2223748686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe234009b30 con 0x7fe23c154fa0 2023-07-06T21:42:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe242b9c700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe23c154fa0 0x7fe23c1f1370 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe234015000 tx=0x7fe234004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234004800 con 0x7fe23c154fa0 2023-07-06T21:42:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234004960 con 0x7fe23c154fa0 2023-07-06T21:42:28.570 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23401b670 con 0x7fe23c154fa0 2023-07-06T21:42:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c1f22a0 con 0x7fe23c154fa0 2023-07-06T21:42:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.563+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c1f2860 con 0x7fe23c154fa0 2023-07-06T21:42:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.564+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c1f2a50 con 0x7fe23c154fa0 2023-07-06T21:42:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.565+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe23401b7d0 con 0x7fe23c154fa0 2023-07-06T21:42:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.566+0000 7fe232ffd700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe22006ace0 0x7fe22006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.566+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe23408a930 con 0x7fe23c154fa0 2023-07-06T21:42:28.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.567+0000 7fe24239b700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe22006ace0 0x7fe22006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.567+0000 7fe24239b700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe22006ace0 0x7fe22006d190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe238009be0 tx=0x7fe2380091c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:28.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.571+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe2340559d0 con 0x7fe23c154fa0 2023-07-06T21:42:28.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.845+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) v1 -- 0x7fe23c1f2100 con 0x7fe23c154fa0 2023-07-06T21:42:28.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.848+0000 7fe232ffd700 1 -- 172.21.15.84:0/2223748686 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v16) v1 ==== 183+0+0 (secure 0 0 0) 0x7fe23c1f2100 con 0x7fe23c154fa0 2023-07-06T21:42:28.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe22006ace0 msgr2=0x7fe22006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:28.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe22006ace0 0x7fe22006d190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe238009be0 tx=0x7fe2380091c0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe23c154fa0 msgr2=0x7fe23c1f1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:28.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe23c154fa0 0x7fe23c1f1370 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe234015000 tx=0x7fe234004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 shutdown_connections 2023-07-06T21:42:28.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe22006ace0 0x7fe22006d190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe23c154fa0 0x7fe23c1f1370 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe23c15fee0 0x7fe23c0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 --2- 172.21.15.84:0/2223748686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe23c160ac0 0x7fe23c0c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:28.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 >> 172.21.15.84:0/2223748686 conn(0x7fe23c0715c0 msgr2=0x7fe23c0cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:28.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 shutdown_connections 2023-07-06T21:42:28.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:28.851+0000 7fe2445ff700 1 -- 172.21.15.84:0/2223748686 wait complete. 2023-07-06T21:42:28.856 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:42:28.873 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:28.874 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:28.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:29.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.148+0000 7fb93ffff700 1 Processor -- start 2023-07-06T21:42:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.148+0000 7fb93ffff700 1 -- start start 2023-07-06T21:42:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.149+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 0x7fb9380cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.149+0000 7fb93ffff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380d0990 con 0x7fb9380d3580 2023-07-06T21:42:29.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.149+0000 7fb93ffff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9380cd9d0 con 0x7fb9380d1580 2023-07-06T21:42:29.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.149+0000 7fb93ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380d3af0 con 0x7fb9380b8530 2023-07-06T21:42:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93d59a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb9380d3580 0x7fb9380d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58672/0 (socket says 172.21.15.84:58672) 2023-07-06T21:42:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93d59a700 1 -- 172.21.15.84:0/3654740511 learned_addr learned my addr 172.21.15.84:0/3654740511 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:29.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93dd9b700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 0x7fb9380cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900346383 0 0) 0x7fb9380d0990 con 0x7fb9380d3580 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9240035b0 con 0x7fb9380d3580 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 703730226 0 0) 0x7fb9240035b0 con 0x7fb9380d3580 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 msgr2=0x7fb9380cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 0x7fb9380cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 >> v1:172.21.15.107:6789/0 conn(0x7fb9380d1580 legacy=0x7fb93815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9380cd410 con 0x7fb9380d3580 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.150+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fb928001670 con 0x7fb9380d3580 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 >> v1:172.21.15.84:6789/0 conn(0x7fb9380d3580 legacy=0x7fb9380d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924003a30 0x7fb924003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb924004850 0x7fb924004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 0x7fb924005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380d3af0 con 0x7fb924004850 2023-07-06T21:42:29.157 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92400baf0 con 0x7fb924003a30 2023-07-06T21:42:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92400bcb0 con 0x7fb9380b8530 2023-07-06T21:42:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb924004850 0x7fb924004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93dd9b700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924003a30 0x7fb924003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93e59c700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 0x7fb924005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 -- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 msgr2=0x7fb924005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:29.158 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9380b8530 0x7fb924005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 -- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924003a30 msgr2=0x7fb924003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924003a30 0x7fb924003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 -- 172.21.15.84:0/3654740511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9240035b0 con 0x7fb924004850 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.151+0000 7fb93d59a700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb924004850 0x7fb924004cc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c160 tx=0x7fb924005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baceedeae392d3bd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928007b70 con 0x7fb924004850 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92800cd20 con 0x7fb924004850 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93cd99700 1 -- 172.21.15.84:0/3654740511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928005ba0 con 0x7fb924004850 2023-07-06T21:42:29.159 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93ffff700 1 -- 172.21.15.84:0/3654740511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb924004850 msgr2=0x7fb924004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93ffff700 1 --2- 172.21.15.84:0/3654740511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb924004850 0x7fb924004cc0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c160 tx=0x7fb924005880 comp rx=0 tx=0).stop 2023-07-06T21:42:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93e59c700 1 -- 172.21.15.84:0/3654740511 reap_dead start 2023-07-06T21:42:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93ffff700 1 -- 172.21.15.84:0/3654740511 shutdown_connections 2023-07-06T21:42:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93ffff700 1 -- 172.21.15.84:0/3654740511 >> 172.21.15.84:0/3654740511 conn(0x7fb9380715c0 msgr2=0x7fb938071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93ffff700 1 -- 172.21.15.84:0/3654740511 shutdown_connections 2023-07-06T21:42:29.160 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.152+0000 7fb93ffff700 1 -- 172.21.15.84:0/3654740511 wait complete. 2023-07-06T21:42:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.154+0000 7fb93ffff700 1 Processor -- start 2023-07-06T21:42:29.162 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.154+0000 7fb93ffff700 1 -- start start 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924004850 0x7fb9381510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 0x7fb9381f1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93ffff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9381f5d00 0x7fb9380b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93ffff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381f5870 con 0x7fb9380c9080 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93ffff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381f5520 con 0x7fb924004850 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93ffff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381f2810 con 0x7fb9381f5d00 2023-07-06T21:42:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93d59a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 0x7fb9381f1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93d59a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 0x7fb9381f1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40446/0 (socket says 172.21.15.84:40446) 2023-07-06T21:42:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93d59a700 1 -- 172.21.15.84:0/2713797051 learned_addr learned my addr 172.21.15.84:0/2713797051 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93e59c700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9381f5d00 0x7fb9380b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.155+0000 7fb93dd9b700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924004850 0x7fb9381510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93d59a700 1 -- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9381f5d00 msgr2=0x7fb9380b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93d59a700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9381f5d00 0x7fb9380b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93d59a700 1 -- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924004850 msgr2=0x7fb9381510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93d59a700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924004850 0x7fb9381510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93d59a700 1 -- 172.21.15.84:0/2713797051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9381f2af0 con 0x7fb9380c9080 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93d59a700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 0x7fb9381f1f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c750 tx=0x7fb928001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928004c80 con 0x7fb9380c9080 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb928004de0 con 0x7fb9380c9080 2023-07-06T21:42:29.165 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9381f2dd0 con 0x7fb9380c9080 2023-07-06T21:42:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928007310 con 0x7fb9380c9080 2023-07-06T21:42:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.156+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9381f2460 con 0x7fb9380c9080 2023-07-06T21:42:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.157+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9380cf490 con 0x7fb9380c9080 2023-07-06T21:42:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.158+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb928019070 con 0x7fb9380c9080 2023-07-06T21:42:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.159+0000 7fb92e7fc700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb91c06ace0 0x7fb91c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.166 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.159+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb92808a3e0 con 0x7fb9380c9080 2023-07-06T21:42:29.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.160+0000 7fb93dd9b700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb91c06ace0 0x7fb91c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.160+0000 7fb93dd9b700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb91c06ace0 0x7fb91c06d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb934009fd0 tx=0x7fb93400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:29.167 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.164+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb928055480 con 0x7fb9380c9080 2023-07-06T21:42:29.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.359+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9380cf1b0 con 0x7fb9380c9080 2023-07-06T21:42:29.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.361+0000 7fb92e7fc700 1 -- 172.21.15.84:0/2713797051 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7fb928058aa0 con 0x7fb9380c9080 2023-07-06T21:42:29.365 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:29.365 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb91c06ace0 msgr2=0x7fb91c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb91c06ace0 0x7fb91c06d190 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb934009fd0 tx=0x7fb93400b040 comp rx=0 tx=0).stop 2023-07-06T21:42:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 msgr2=0x7fb9381f1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 0x7fb9381f1f20 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fb92800c750 tx=0x7fb928001930 comp rx=0 tx=0).stop 2023-07-06T21:42:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 shutdown_connections 2023-07-06T21:42:29.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb91c06ace0 0x7fb91c06d190 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9381f5d00 0x7fb9380b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb924004850 0x7fb9381510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 --2- 172.21.15.84:0/2713797051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9380c9080 0x7fb9381f1f20 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 >> 172.21.15.84:0/2713797051 conn(0x7fb9380715c0 msgr2=0x7fb9380b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:29.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 shutdown_connections 2023-07-06T21:42:29.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.363+0000 7fb93ffff700 1 -- 172.21.15.84:0/2713797051 wait complete. 2023-07-06T21:42:29.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:42:08.774391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:42:09.558402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:29.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:29.583 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.580+0000 7f166ebff700 1 Processor -- start 2023-07-06T21:42:29.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.581+0000 7f166ebff700 1 -- start start 2023-07-06T21:42:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f166ebff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f16680b81c0 0x7f16680cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f166ebff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16680d1830 con 0x7f16680b1a80 2023-07-06T21:42:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f166ebff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1668160d40 con 0x7f16680ceda0 2023-07-06T21:42:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f166ebff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16680cf330 con 0x7f16680b81c0 2023-07-06T21:42:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f1667fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f16680b1a80 0x7f166815ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58676/0 (socket says 172.21.15.84:58676) 2023-07-06T21:42:29.585 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f1667fff700 1 -- 172.21.15.84:0/2550027440 learned_addr learned my addr 172.21.15.84:0/2550027440 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481412373 0 0) 0x7f16680d1830 con 0x7f16680b1a80 2023-07-06T21:42:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f166c99b700 1 --2- 172.21.15.84:0/2550027440 >> v2:172.21.15.107:3300/0 conn(0x7f16680b81c0 0x7f16680cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.588 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.582+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16580035b0 con 0x7f16680b1a80 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961837191 0 0) 0x7f16580035b0 con 0x7f16680b1a80 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 >> v2:172.21.15.107:3300/0 conn(0x7f16680b81c0 msgr2=0x7f16680cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 --2- 172.21.15.84:0/2550027440 >> v2:172.21.15.107:3300/0 conn(0x7f16680b81c0 0x7f16680cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 >> v1:172.21.15.107:6789/0 conn(0x7f16680ceda0 legacy=0x7f1668160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680d1270 con 0x7f16680b1a80 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f1654001670 con 0x7f16680b1a80 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 >> v1:172.21.15.84:6789/0 conn(0x7f16680b1a80 legacy=0x7f166815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.589 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1658003a00 0x7f1658003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658004850 0x7f1658004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1658005200 0x7f165800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16680cf330 con 0x7f1658005200 2023-07-06T21:42:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165800c080 con 0x7f1658004850 2023-07-06T21:42:29.590 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165800c240 con 0x7f1658003a00 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f166d19c700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1658005200 0x7f165800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f166c99b700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1658003a00 0x7f1658003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.583+0000 7f1667fff700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658004850 0x7f1658004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 -- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1658003a00 msgr2=0x7f1658003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1658003a00 0x7f1658003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 -- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658004850 msgr2=0x7f1658004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:29.591 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658004850 0x7f1658004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 -- 172.21.15.84:0/2550027440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16580035b0 con 0x7f1658005200 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 -- 172.21.15.84:0/2550027440 reap_dead start 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f166d19c700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1658005200 0x7f165800ba10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f166000c190 tx=0x7f1658005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b407fcd05ebb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660001db0 con 0x7f1658005200 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166000ccd0 con 0x7f1658005200 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.584+0000 7f16677fe700 1 -- 172.21.15.84:0/2550027440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1660003c40 con 0x7f1658005200 2023-07-06T21:42:29.592 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 -- 172.21.15.84:0/2550027440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1658005200 msgr2=0x7f165800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1658005200 0x7f165800ba10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f166000c190 tx=0x7f1658005880 comp rx=0 tx=0).stop 2023-07-06T21:42:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 -- 172.21.15.84:0/2550027440 shutdown_connections 2023-07-06T21:42:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 --2- 172.21.15.84:0/2550027440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1658005200 0x7f165800ba10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 -- 172.21.15.84:0/2550027440 >> 172.21.15.84:0/2550027440 conn(0x7f166806b2b0 msgr2=0x7f166806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 -- 172.21.15.84:0/2550027440 shutdown_connections 2023-07-06T21:42:29.593 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.585+0000 7f166ebff700 1 -- 172.21.15.84:0/2550027440 wait complete. 2023-07-06T21:42:29.595 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.586+0000 7f166ebff700 1 Processor -- start 2023-07-06T21:42:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.587+0000 7f166ebff700 1 -- start start 2023-07-06T21:42:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166ebff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658005200 0x7f16681ef290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166ebff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 0x7f16681f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166ebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16681f3430 0x7f16680c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166ebff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16681ef7d0 con 0x7f16681f2760 2023-07-06T21:42:29.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166ebff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16680c6a30 con 0x7f1658005200 2023-07-06T21:42:29.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166ebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16680c6d10 con 0x7f16681f3430 2023-07-06T21:42:29.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 0x7f16681f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 0x7f16681f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40460/0 (socket says 172.21.15.84:40460) 2023-07-06T21:42:29.597 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 -- 172.21.15.84:0/2353034816 learned_addr learned my addr 172.21.15.84:0/2353034816 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166d19c700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16681f3430 0x7f16680c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166c99b700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658005200 0x7f16681ef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 -- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16681f3430 msgr2=0x7f16680c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16681f3430 0x7f16680c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 -- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658005200 msgr2=0x7f16681ef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658005200 0x7f16681ef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 -- 172.21.15.84:0/2353034816 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680c6ff0 con 0x7f16681f2760 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166c99b700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658005200 0x7f16681ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f166d19c700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16681f3430 0x7f16680c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.588+0000 7f1667fff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 0x7f16681f2ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1654007290 tx=0x7f16540075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.589+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400f660 con 0x7f16681f2760 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.589+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f165400f7c0 con 0x7f16681f2760 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.589+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f165400fb70 con 0x7f16681f2760 2023-07-06T21:42:29.599 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.589+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16680c72d0 con 0x7f16681f2760 2023-07-06T21:42:29.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.589+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16680c6720 con 0x7f16681f2760 2023-07-06T21:42:29.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.590+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16680c77f0 con 0x7f16681f2760 2023-07-06T21:42:29.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.591+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f165400fcd0 con 0x7f16681f2760 2023-07-06T21:42:29.600 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.593+0000 7f16657fa700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f164c06ace0 0x7f164c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:29.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.593+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f16540894f0 con 0x7f16681f2760 2023-07-06T21:42:29.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.593+0000 7f166c99b700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f164c06ace0 0x7f164c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:29.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.593+0000 7f166c99b700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f164c06ace0 0x7f164c06d190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f165c009470 tx=0x7f165c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:29.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.597+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f16680c77f0 con 0x7f16681f2760 2023-07-06T21:42:29.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.791+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f16680c77f0 con 0x7f16681f2760 2023-07-06T21:42:29.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.791+0000 7f16657fa700 1 -- 172.21.15.84:0/2353034816 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16680c77f0 con 0x7f16681f2760 2023-07-06T21:42:29.794 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:29.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f164c06ace0 msgr2=0x7f164c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f164c06ace0 0x7f164c06d190 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f165c009470 tx=0x7f165c008040 comp rx=0 tx=0).stop 2023-07-06T21:42:29.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 msgr2=0x7f16681f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:29.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 0x7f16681f2ef0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1654007290 tx=0x7f16540075a0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 shutdown_connections 2023-07-06T21:42:29.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f164c06ace0 0x7f164c06d190 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16681f3430 0x7f16680c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1658005200 0x7f16681ef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 --2- 172.21.15.84:0/2353034816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16681f2760 0x7f16681f2ef0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:29.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.793+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 >> 172.21.15.84:0/2353034816 conn(0x7f166806b2b0 msgr2=0x7f16680734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:29.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.794+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 shutdown_connections 2023-07-06T21:42:29.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:29.794+0000 7f166ebff700 1 -- 172.21.15.84:0/2353034816 wait complete. 2023-07-06T21:42:29.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.009+0000 7f83bf1ff700 1 Processor -- start 2023-07-06T21:42:30.013 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.009+0000 7f83bf1ff700 1 -- start start 2023-07-06T21:42:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.010+0000 7f83bf1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b8069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.010+0000 7f83bf1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83b806a3d0 0x7f83b806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.010+0000 7f83bf1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b806a830 con 0x7f83b815c020 2023-07-06T21:42:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.010+0000 7f83bf1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b815d400 con 0x7f83b806a3d0 2023-07-06T21:42:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bf1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b806cf40 con 0x7f83b80c90a0 2023-07-06T21:42:30.014 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83b806a3d0 0x7f83b806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f83b806a3d0 0x7f83b806d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40464/0 (socket says 172.21.15.84:40464) 2023-07-06T21:42:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 -- 172.21.15.84:0/853726947 learned_addr learned my addr 172.21.15.84:0/853726947 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bcf9b700 1 --1- 172.21.15.84:0/853726947 >> v1:172.21.15.107:6789/0 conn(0x7f83b815c020 0x7f83b815c460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:54536/0 (socket says 172.21.15.84:54536) 2023-07-06T21:42:30.015 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83b7fff700 1 --2- 172.21.15.84:0/853726947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b8069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 -- 172.21.15.84:0/853726947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 msgr2=0x7f83b8069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 --2- 172.21.15.84:0/853726947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b8069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 -- 172.21.15.84:0/853726947 >> v1:172.21.15.107:6789/0 conn(0x7f83b815c020 legacy=0x7f83b815c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.016 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 -- 172.21.15.84:0/853726947 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b815cb70 con 0x7f83b806a3d0 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.011+0000 7f83bd79c700 1 --2- 172.21.15.84:0/853726947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b806d3d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f83b000d150 tx=0x7f83b000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b576375df283e27f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83b77fe700 1 -- 172.21.15.84:0/853726947 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0010070 con 0x7f83b806a3d0 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83b77fe700 1 -- 172.21.15.84:0/853726947 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b000dd10 con 0x7f83b806a3d0 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83b77fe700 1 -- 172.21.15.84:0/853726947 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000b310 con 0x7f83b806a3d0 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 -- 172.21.15.84:0/853726947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 msgr2=0x7f83b806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/853726947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b806d3d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f83b000d150 tx=0x7f83b000d460 comp rx=0 tx=0).stop 2023-07-06T21:42:30.017 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 -- 172.21.15.84:0/853726947 shutdown_connections 2023-07-06T21:42:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/853726947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b806d3d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/853726947 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b8069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 -- 172.21.15.84:0/853726947 >> 172.21.15.84:0/853726947 conn(0x7f83b80cbf10 msgr2=0x7f83b80ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 -- 172.21.15.84:0/853726947 shutdown_connections 2023-07-06T21:42:30.018 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.012+0000 7f83bf1ff700 1 -- 172.21.15.84:0/853726947 wait complete. 2023-07-06T21:42:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.014+0000 7f83bf1ff700 1 Processor -- start 2023-07-06T21:42:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.014+0000 7f83bf1ff700 1 -- start start 2023-07-06T21:42:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bf1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b80b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bf1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b80c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bf1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83b815c020 0x7f83b80b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.020 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bf1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80c3300 con 0x7f83b806a3d0 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bf1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80c3550 con 0x7f83b815c020 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bf1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80c3830 con 0x7f83b80c90a0 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.015+0000 7f83bcf9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b80b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b80b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40478/0 (socket says 172.21.15.84:40478) 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 -- 172.21.15.84:0/2836579208 learned_addr learned my addr 172.21.15.84:0/2836579208 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83b7fff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b80c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bd79c700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83b815c020 0x7f83b80b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.021 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 -- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 msgr2=0x7f83b80c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b80c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 -- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83b815c020 msgr2=0x7f83b80b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83b815c020 0x7f83b80b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 -- 172.21.15.84:0/2836579208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b000bd40 con 0x7f83b806a3d0 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bcf9b700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b80b2170 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c520 tx=0x7f83ac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac005cb0 con 0x7f83b806a3d0 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ac005e10 con 0x7f83b806a3d0 2023-07-06T21:42:30.022 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b80c3b10 con 0x7f83b806a3d0 2023-07-06T21:42:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.016+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0072d0 con 0x7f83b806a3d0 2023-07-06T21:42:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.017+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b80c40d0 con 0x7f83b806a3d0 2023-07-06T21:42:30.023 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.018+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b804e1d0 con 0x7f83b806a3d0 2023-07-06T21:42:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.018+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f83ac007430 con 0x7f83b806a3d0 2023-07-06T21:42:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.020+0000 7f83b57fa700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f839c06ace0 0x7f839c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.020+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f83ac088730 con 0x7f83b806a3d0 2023-07-06T21:42:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.020+0000 7f83b7fff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f839c06ace0 0x7f839c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.024 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.020+0000 7f83b7fff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f839c06ace0 0x7f839c06d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83b80cd130 tx=0x7f83a4009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.026 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.024+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f83ac053880 con 0x7f83b806a3d0 2023-07-06T21:42:30.223 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.220+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83b80acc70 con 0x7f83b806a3d0 2023-07-06T21:42:30.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.221+0000 7f83b57fa700 1 -- 172.21.15.84:0/2836579208 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f83ac056ea0 con 0x7f83b806a3d0 2023-07-06T21:42:30.224 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f839c06ace0 msgr2=0x7f839c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f839c06ace0 0x7f839c06d190 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f83b80cd130 tx=0x7f83a4009210 comp rx=0 tx=0).stop 2023-07-06T21:42:30.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 msgr2=0x7f83b80b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b80b2170 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f83ac00c520 tx=0x7f83ac00c830 comp rx=0 tx=0).stop 2023-07-06T21:42:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 shutdown_connections 2023-07-06T21:42:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f839c06ace0 0x7f839c06d190 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83b815c020 0x7f83b80b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83b806a3d0 0x7f83b80b2170 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.227 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 --2- 172.21.15.84:0/2836579208 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83b80c90a0 0x7f83b80c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 >> 172.21.15.84:0/2836579208 conn(0x7f83b80cbf10 msgr2=0x7f83b80cab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 shutdown_connections 2023-07-06T21:42:30.228 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.223+0000 7f83bf1ff700 1 -- 172.21.15.84:0/2836579208 wait complete. 2023-07-06T21:42:30.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:42:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.438+0000 7f6e721ff700 1 Processor -- start 2023-07-06T21:42:30.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.438+0000 7f6e721ff700 1 -- start start 2023-07-06T21:42:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.439+0000 7f6e721ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e6c159220 0x7f6e6c0ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.439+0000 7f6e721ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e6c070ab0 0x7f6e6c0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.439+0000 7f6e721ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c0cf600 con 0x7f6e6c0cc3d0 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.439+0000 7f6e721ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c0ccf40 con 0x7f6e6c159220 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.439+0000 7f6e721ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c0c9b00 con 0x7f6e6c070ab0 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e6c159220 0x7f6e6c0ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f6e6c159220 0x7f6e6c0ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40482/0 (socket says 172.21.15.84:40482) 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 -- 172.21.15.84:0/2868884187 learned_addr learned my addr 172.21.15.84:0/2868884187 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6bfff700 1 --2- 172.21.15.84:0/2868884187 >> v2:172.21.15.107:3300/0 conn(0x7f6e6c070ab0 0x7f6e6c0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 -- 172.21.15.84:0/2868884187 >> v2:172.21.15.107:3300/0 conn(0x7f6e6c070ab0 msgr2=0x7f6e6c0cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 --2- 172.21.15.84:0/2868884187 >> v2:172.21.15.107:3300/0 conn(0x7f6e6c070ab0 0x7f6e6c0cc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 -- 172.21.15.84:0/2868884187 >> v1:172.21.15.107:6789/0 conn(0x7f6e6c0cc3d0 legacy=0x7f6e6c0703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 -- 172.21.15.84:0/2868884187 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c0c9de0 con 0x7f6e6c159220 2023-07-06T21:42:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6bfff700 1 --2- 172.21.15.84:0/2868884187 >> v2:172.21.15.107:3300/0 conn(0x7f6e6c070ab0 0x7f6e6c0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:30.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.440+0000 7f6e6b7fe700 1 --2- 172.21.15.84:0/2868884187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0ca8c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c070330 tx=0x7f6e5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a185bdca22a0c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e6a7fc700 1 -- 172.21.15.84:0/2868884187 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00b870 con 0x7f6e6c159220 2023-07-06T21:42:30.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e6a7fc700 1 -- 172.21.15.84:0/2868884187 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c00b9d0 con 0x7f6e6c159220 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e6a7fc700 1 -- 172.21.15.84:0/2868884187 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c00bd60 con 0x7f6e6c159220 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 -- 172.21.15.84:0/2868884187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 msgr2=0x7f6e6c0ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 --2- 172.21.15.84:0/2868884187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0ca8c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c070330 tx=0x7f6e5c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 -- 172.21.15.84:0/2868884187 shutdown_connections 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 --2- 172.21.15.84:0/2868884187 >> v2:172.21.15.107:3300/0 conn(0x7f6e6c070ab0 0x7f6e6c0cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 --2- 172.21.15.84:0/2868884187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0ca8c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 -- 172.21.15.84:0/2868884187 >> 172.21.15.84:0/2868884187 conn(0x7f6e6c0cbf10 msgr2=0x7f6e6c0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:30.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 -- 172.21.15.84:0/2868884187 shutdown_connections 2023-07-06T21:42:30.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.441+0000 7f6e721ff700 1 -- 172.21.15.84:0/2868884187 wait complete. 2023-07-06T21:42:30.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.443+0000 7f6e721ff700 1 Processor -- start 2023-07-06T21:42:30.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.443+0000 7f6e721ff700 1 -- start start 2023-07-06T21:42:30.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e721ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e6c070ab0 0x7f6e6c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e721ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e6c0cc3d0 0x7f6e6c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e721ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e721ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15bcb0 con 0x7f6e6c159220 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e721ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15b0e0 con 0x7f6e6c0cc3d0 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e721ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c15b380 con 0x7f6e6c070ab0 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40488/0 (socket says 172.21.15.84:40488) 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6bfff700 1 -- 172.21.15.84:0/935819056 learned_addr learned my addr 172.21.15.84:0/935819056 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6affd700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e6c0cc3d0 0x7f6e6c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6b7fe700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e6c070ab0 0x7f6e6c154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6bfff700 1 -- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e6c070ab0 msgr2=0x7f6e6c154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6bfff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e6c070ab0 0x7f6e6c154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.444+0000 7f6e6bfff700 1 -- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e6c0cc3d0 msgr2=0x7f6e6c0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e6bfff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e6c0cc3d0 0x7f6e6c0aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e6bfff700 1 -- 172.21.15.84:0/935819056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c009550 con 0x7f6e6c159220 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e6affd700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e6c0cc3d0 0x7f6e6c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e6bfff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0b2680 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009fa0 tx=0x7f6e6400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003a20 con 0x7f6e6c159220 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6400f040 con 0x7f6e6c159220 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c15b660 con 0x7f6e6c159220 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64003de0 con 0x7f6e6c159220 2023-07-06T21:42:30.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.445+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c1550a0 con 0x7f6e6c159220 2023-07-06T21:42:30.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.446+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c15be20 con 0x7f6e6c159220 2023-07-06T21:42:30.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.447+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6e64004030 con 0x7f6e6c159220 2023-07-06T21:42:30.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.448+0000 7f6e53fff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6e5806ace0 0x7f6e5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.448+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e64089740 con 0x7f6e6c159220 2023-07-06T21:42:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.448+0000 7f6e6b7fe700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6e5806ace0 0x7f6e5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.449+0000 7f6e6b7fe700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6e5806ace0 0x7f6e5806d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c014040 tx=0x7f6e5c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.452+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6e64054750 con 0x7f6e6c159220 2023-07-06T21:42:30.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.648+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e6c06c380 con 0x7f6e6c159220 2023-07-06T21:42:30.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.648+0000 7f6e53fff700 1 -- 172.21.15.84:0/935819056 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6e64057d70 con 0x7f6e6c159220 2023-07-06T21:42:30.652 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:42:30.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6e5806ace0 msgr2=0x7f6e5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6e5806ace0 0x7f6e5806d190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c014040 tx=0x7f6e5c008030 comp rx=0 tx=0).stop 2023-07-06T21:42:30.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 msgr2=0x7f6e6c0b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0b2680 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6e64009fa0 tx=0x7f6e6400d460 comp rx=0 tx=0).stop 2023-07-06T21:42:30.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 shutdown_connections 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f6e5806ace0 0x7f6e5806d190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e6c159220 0x7f6e6c0b2680 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e6c070ab0 0x7f6e6c154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 --2- 172.21.15.84:0/935819056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e6c0cc3d0 0x7f6e6c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 >> 172.21.15.84:0/935819056 conn(0x7f6e6c0cbf10 msgr2=0x7f6e6c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 shutdown_connections 2023-07-06T21:42:30.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.651+0000 7f6e721ff700 1 -- 172.21.15.84:0/935819056 wait complete. 2023-07-06T21:42:30.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:42:30.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.866+0000 7f10373ff700 1 Processor -- start 2023-07-06T21:42:30.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.866+0000 7f10373ff700 1 -- start start 2023-07-06T21:42:30.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f10373ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f10300ac450 0x7f10300ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f10373ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f10300ad960 0x7f10300add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f10373ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10300cf690 con 0x7f103015c750 2023-07-06T21:42:30.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f10373ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300ae2a0 con 0x7f10300ad960 2023-07-06T21:42:30.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f10373ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300ac8c0 con 0x7f10300ac450 2023-07-06T21:42:30.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f103519b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f103015c750 0x7f10300c9e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58678/0 (socket says 172.21.15.84:58678) 2023-07-06T21:42:30.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f103519b700 1 -- 172.21.15.84:0/146978688 learned_addr learned my addr 172.21.15.84:0/146978688 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:30.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f103599c700 1 --2- 172.21.15.84:0/146978688 >> v2:172.21.15.84:3300/0 conn(0x7f10300ad960 0x7f10300add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f103499a700 1 --2- 172.21.15.84:0/146978688 >> v2:172.21.15.107:3300/0 conn(0x7f10300ac450 0x7f10300ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f1027fff700 1 -- 172.21.15.84:0/146978688 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645183222 0 0) 0x7f10300cf690 con 0x7f103015c750 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f1027fff700 1 -- 172.21.15.84:0/146978688 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f101c0035b0 con 0x7f103015c750 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.868+0000 7f103599c700 1 -- 172.21.15.84:0/146978688 >> v2:172.21.15.107:3300/0 conn(0x7f10300ac450 msgr2=0x7f10300ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f103599c700 1 --2- 172.21.15.84:0/146978688 >> v2:172.21.15.107:3300/0 conn(0x7f10300ac450 0x7f10300ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f103599c700 1 -- 172.21.15.84:0/146978688 >> v1:172.21.15.84:6789/0 conn(0x7f103015c750 legacy=0x7f10300c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f103599c700 1 -- 172.21.15.84:0/146978688 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300acba0 con 0x7f10300ad960 2023-07-06T21:42:30.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f103599c700 1 --2- 172.21.15.84:0/146978688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300add60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f10300c32d0 tx=0x7f102800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63770842a9a418fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f1027fff700 1 -- 172.21.15.84:0/146978688 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102800e070 con 0x7f10300ad960 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f1027fff700 1 -- 172.21.15.84:0/146978688 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102800bb20 con 0x7f10300ad960 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f1027fff700 1 -- 172.21.15.84:0/146978688 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1028012430 con 0x7f10300ad960 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f10373ff700 1 -- 172.21.15.84:0/146978688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 msgr2=0x7f10300add60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.869+0000 7f10373ff700 1 --2- 172.21.15.84:0/146978688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300add60 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f10300c32d0 tx=0x7f102800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.870+0000 7f10373ff700 1 -- 172.21.15.84:0/146978688 shutdown_connections 2023-07-06T21:42:30.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.870+0000 7f10373ff700 1 --2- 172.21.15.84:0/146978688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300add60 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.870+0000 7f10373ff700 1 --2- 172.21.15.84:0/146978688 >> v2:172.21.15.107:3300/0 conn(0x7f10300ac450 0x7f10300ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.870+0000 7f10373ff700 1 -- 172.21.15.84:0/146978688 >> 172.21.15.84:0/146978688 conn(0x7f103006b8c0 msgr2=0x7f10300cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.870+0000 7f10373ff700 1 -- 172.21.15.84:0/146978688 shutdown_connections 2023-07-06T21:42:30.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.870+0000 7f10373ff700 1 -- 172.21.15.84:0/146978688 wait complete. 2023-07-06T21:42:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.871+0000 7f10373ff700 1 Processor -- start 2023-07-06T21:42:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.871+0000 7f10373ff700 1 -- start start 2023-07-06T21:42:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.872+0000 7f10373ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10300ac450 0x7f1030155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.872+0000 7f10373ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.872+0000 7f10373ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f103015c750 0x7f10300b2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.872+0000 7f10373ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300c5f80 con 0x7f10300ad960 2023-07-06T21:42:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.872+0000 7f10373ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300c6260 con 0x7f10300ac450 2023-07-06T21:42:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.872+0000 7f10373ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10300c7a50 con 0x7f103015c750 2023-07-06T21:42:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40504/0 (socket says 172.21.15.84:40504) 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 -- 172.21.15.84:0/743007963 learned_addr learned my addr 172.21.15.84:0/743007963 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 -- 172.21.15.84:0/743007963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f103015c750 msgr2=0x7f10300b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f103015c750 0x7f10300b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 -- 172.21.15.84:0/743007963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10300ac450 msgr2=0x7f1030155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10300ac450 0x7f1030155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 -- 172.21.15.84:0/743007963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028009a70 con 0x7f10300ad960 2023-07-06T21:42:30.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f103499a700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300c5910 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9b0 tx=0x7f102000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020009710 con 0x7f10300ad960 2023-07-06T21:42:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102000c850 con 0x7f10300ad960 2023-07-06T21:42:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10300c7d30 con 0x7f10300ad960 2023-07-06T21:42:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.873+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000cbc0 con 0x7f10300ad960 2023-07-06T21:42:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.874+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10300c9700 con 0x7f10300ad960 2023-07-06T21:42:30.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.875+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10300c6480 con 0x7f10300ad960 2023-07-06T21:42:30.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.876+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f102000cd20 con 0x7f10300ad960 2023-07-06T21:42:30.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.877+0000 7f1025ffb700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:30.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.877+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1020089500 con 0x7f10300ad960 2023-07-06T21:42:30.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.877+0000 7f103519b700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:30.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.878+0000 7f103519b700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f101406ace0 0x7f101406d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f102c001360 tx=0x7f102c007850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:30.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:30.881+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f10200580d0 con 0x7f10300ad960 2023-07-06T21:42:31.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.079+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f103004e1d0 con 0x7f10300ad960 2023-07-06T21:42:31.082 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.079+0000 7f1025ffb700 1 -- 172.21.15.84:0/743007963 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1020057c60 con 0x7f10300ad960 2023-07-06T21:42:31.082 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:42:31.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f101406ace0 msgr2=0x7f101406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f101406ace0 0x7f101406d190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f102c001360 tx=0x7f102c007850 comp rx=0 tx=0).stop 2023-07-06T21:42:31.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 msgr2=0x7f10300c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.086 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300c5910 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f102000e9b0 tx=0x7f102000ed70 comp rx=0 tx=0).stop 2023-07-06T21:42:31.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 shutdown_connections 2023-07-06T21:42:31.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f101406ace0 0x7f101406d190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f103015c750 0x7f10300b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10300ac450 0x7f1030155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.087 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 --2- 172.21.15.84:0/743007963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10300ad960 0x7f10300c5910 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.081+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 >> 172.21.15.84:0/743007963 conn(0x7f103006b8c0 msgr2=0x7f10300b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:31.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.082+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 shutdown_connections 2023-07-06T21:42:31.088 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.082+0000 7f10373ff700 1 -- 172.21.15.84:0/743007963 wait complete. 2023-07-06T21:42:31.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:42:31.297 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.294+0000 7f33ff3ff700 1 Processor -- start 2023-07-06T21:42:31.298 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.294+0000 7f33ff3ff700 1 -- start start 2023-07-06T21:42:31.300 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.295+0000 7f33ff3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33f80b83c0 0x7f33f80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.295+0000 7f33ff3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.295+0000 7f33ff3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80cfee0 con 0x7f33f80cfa90 2023-07-06T21:42:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.295+0000 7f33ff3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80d0a80 con 0x7f33f80b83c0 2023-07-06T21:42:31.301 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.295+0000 7f33ff3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80b8790 con 0x7f33f80d3710 2023-07-06T21:42:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33f80b83c0 0x7f33f80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f33f80b83c0 0x7f33f80cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40512/0 (socket says 172.21.15.84:40512) 2023-07-06T21:42:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd99c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f33f80cfa90 0x7f33f815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:58682/0 (socket says 172.21.15.84:58682) 2023-07-06T21:42:31.302 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 -- 172.21.15.84:0/3619247541 learned_addr learned my addr 172.21.15.84:0/3619247541 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fc99a700 1 --2- 172.21.15.84:0/3619247541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 -- 172.21.15.84:0/3619247541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 msgr2=0x7f33f80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 --2- 172.21.15.84:0/3619247541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 -- 172.21.15.84:0/3619247541 >> v1:172.21.15.84:6789/0 conn(0x7f33f80cfa90 legacy=0x7f33f815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.303 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 -- 172.21.15.84:0/3619247541 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80b4e70 con 0x7f33f80b83c0 2023-07-06T21:42:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33effff700 1 -- 172.21.15.84:0/3619247541 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988056577 0 0) 0x7f33f0001490 con 0x7f33f80cfa90 2023-07-06T21:42:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.296+0000 7f33fd19b700 1 --2- 172.21.15.84:0/3619247541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80b83c0 0x7f33f80cd200 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f33f40098f0 tx=0x7f33f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2cd49dc1c3920 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33effff700 1 -- 172.21.15.84:0/3619247541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400b8a0 con 0x7f33f80b83c0 2023-07-06T21:42:31.304 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33effff700 1 -- 172.21.15.84:0/3619247541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f400ba00 con 0x7f33f80b83c0 2023-07-06T21:42:31.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33effff700 1 -- 172.21.15.84:0/3619247541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f400bd90 con 0x7f33f80b83c0 2023-07-06T21:42:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 -- 172.21.15.84:0/3619247541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80b83c0 msgr2=0x7f33f80cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/3619247541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80b83c0 0x7f33f80cd200 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f33f40098f0 tx=0x7f33f4009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 -- 172.21.15.84:0/3619247541 shutdown_connections 2023-07-06T21:42:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/3619247541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80b83c0 0x7f33f80cd200 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/3619247541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 -- 172.21.15.84:0/3619247541 >> 172.21.15.84:0/3619247541 conn(0x7f33f806b350 msgr2=0x7f33f806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:31.307 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 -- 172.21.15.84:0/3619247541 shutdown_connections 2023-07-06T21:42:31.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.297+0000 7f33ff3ff700 1 -- 172.21.15.84:0/3619247541 wait complete. 2023-07-06T21:42:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.299+0000 7f33ff3ff700 1 Processor -- start 2023-07-06T21:42:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.299+0000 7f33ff3ff700 1 -- start start 2023-07-06T21:42:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33ff3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b83c0 0x7f33f81ef3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33ff3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 0x7f33f80c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33ff3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f8154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33ff3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f815ca40 con 0x7f33f80cfa90 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33ff3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c75d0 con 0x7f33f80b83c0 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33ff3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c8010 con 0x7f33f80d3710 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33fc99a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 0x7f33f80c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33fc99a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 0x7f33f80c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40520/0 (socket says 172.21.15.84:40520) 2023-07-06T21:42:31.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33fc99a700 1 -- 172.21.15.84:0/116116086 learned_addr learned my addr 172.21.15.84:0/116116086 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33fd19b700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b83c0 0x7f33f81ef3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.300+0000 7f33fd99c700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f8154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33fc99a700 1 -- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 msgr2=0x7f33f8154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33fc99a700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f8154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33fc99a700 1 -- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b83c0 msgr2=0x7f33f81ef3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33fc99a700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b83c0 0x7f33f81ef3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.313 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33fc99a700 1 -- 172.21.15.84:0/116116086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4009550 con 0x7f33f80cfa90 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33fc99a700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 0x7f33f80c8dd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f33e800d800 tx=0x7f33e800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e80126b0 con 0x7f33f80cfa90 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e8012810 con 0x7f33f80cfa90 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e8012ba0 con 0x7f33f80cfa90 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f80c82f0 con 0x7f33f80cfa90 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.301+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f80c88b0 con 0x7f33f80cfa90 2023-07-06T21:42:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.303+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f80b6a90 con 0x7f33f80cfa90 2023-07-06T21:42:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.303+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f33e8012d00 con 0x7f33f80cfa90 2023-07-06T21:42:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.304+0000 7f33edffb700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f33dc06ace0 0x7f33dc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.304+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33e8089550 con 0x7f33f80cfa90 2023-07-06T21:42:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.305+0000 7f33fd19b700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f33dc06ace0 0x7f33dc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.305+0000 7f33fd19b700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f33dc06ace0 0x7f33dc06d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f33f4014040 tx=0x7f33f4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:31.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.309+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f33e8058090 con 0x7f33f80cfa90 2023-07-06T21:42:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.521+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f33f80b67b0 con 0x7f33f80cfa90 2023-07-06T21:42:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.521+0000 7f33edffb700 1 -- 172.21.15.84:0/116116086 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f33e8057c20 con 0x7f33f80cfa90 2023-07-06T21:42:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f33dc06ace0 msgr2=0x7f33dc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f33dc06ace0 0x7f33dc06d190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f33f4014040 tx=0x7f33f4008030 comp rx=0 tx=0).stop 2023-07-06T21:42:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 msgr2=0x7f33f80c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 0x7f33f80c8dd0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f33e800d800 tx=0x7f33e800db10 comp rx=0 tx=0).stop 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 shutdown_connections 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f33dc06ace0 0x7f33dc06d190 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d3710 0x7f33f8154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b83c0 0x7f33f81ef3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 --2- 172.21.15.84:0/116116086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f33f80cfa90 0x7f33f80c8dd0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.523+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 >> 172.21.15.84:0/116116086 conn(0x7f33f806b350 msgr2=0x7f33f80d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.524+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 shutdown_connections 2023-07-06T21:42:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.524+0000 7f33ff3ff700 1 -- 172.21.15.84:0/116116086 wait complete. 2023-07-06T21:42:31.529 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:42:31.545 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:31.545 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:31.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.817+0000 7f9acebff700 1 Processor -- start 2023-07-06T21:42:31.820 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.817+0000 7f9acebff700 1 -- start start 2023-07-06T21:42:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acebff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9ac80cfa90 0x7f9ac80d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acebff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80cd910 con 0x7f9ac80b83c0 2023-07-06T21:42:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acebff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80cfee0 con 0x7f9ac80cfa90 2023-07-06T21:42:31.821 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80d0a80 con 0x7f9ac80d3710 2023-07-06T21:42:31.822 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acc99b700 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f9ac80b83c0 0x7f9ac80cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.84:54552/0 (socket says 172.21.15.84:54552) 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acc99b700 1 -- 172.21.15.84:0/1358337575 learned_addr learned my addr 172.21.15.84:0/1358337575 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.818+0000 7f9acd19c700 1 --2- 172.21.15.84:0/1358337575 >> v2:172.21.15.107:3300/0 conn(0x7f9ac80cfa90 0x7f9ac80d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9acd19c700 1 -- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 msgr2=0x7f9ac80cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9ac77fe700 1 -- 172.21.15.84:0/1358337575 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964188685 0 0) 0x7f9abc001110 con 0x7f9ac80b83c0 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9acd19c700 1 --2- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac80cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9acd19c700 1 -- 172.21.15.84:0/1358337575 >> v1:172.21.15.107:6789/0 conn(0x7f9ac80b83c0 legacy=0x7f9ac80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.823 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9acd19c700 1 -- 172.21.15.84:0/1358337575 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80b8790 con 0x7f9ac80cfa90 2023-07-06T21:42:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9ac7fff700 1 --2- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9acd19c700 1 --2- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80cfa90 0x7f9ac80d0f10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009d90 tx=0x7f9ac000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e0110ea37ff788 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9ac77fe700 1 -- 172.21.15.84:0/1358337575 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000e070 con 0x7f9ac80cfa90 2023-07-06T21:42:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9ac77fe700 1 -- 172.21.15.84:0/1358337575 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac000bb20 con 0x7f9ac80cfa90 2023-07-06T21:42:31.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.819+0000 7f9ac77fe700 1 -- 172.21.15.84:0/1358337575 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0012430 con 0x7f9ac80cfa90 2023-07-06T21:42:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 -- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80cfa90 msgr2=0x7f9ac80d0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 --2- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80cfa90 0x7f9ac80d0f10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0009d90 tx=0x7f9ac000b330 comp rx=0 tx=0).stop 2023-07-06T21:42:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 -- 172.21.15.84:0/1358337575 shutdown_connections 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 --2- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80cfa90 0x7f9ac80d0f10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 --2- 172.21.15.84:0/1358337575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac80cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 -- 172.21.15.84:0/1358337575 >> 172.21.15.84:0/1358337575 conn(0x7f9ac806b350 msgr2=0x7f9ac806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 -- 172.21.15.84:0/1358337575 shutdown_connections 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.820+0000 7f9acebff700 1 -- 172.21.15.84:0/1358337575 wait complete. 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.822+0000 7f9acebff700 1 Processor -- start 2023-07-06T21:42:31.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.822+0000 7f9acebff700 1 -- start start 2023-07-06T21:42:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acebff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80b83c0 0x7f9ac80c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acebff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 0x7f9ac815f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acebff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac8149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acebff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815de10 con 0x7f9ac80cfa90 2023-07-06T21:42:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acebff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815e7a0 con 0x7f9ac80b83c0 2023-07-06T21:42:31.828 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acebff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac815e060 con 0x7f9ac80d3710 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 0x7f9ac815f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 0x7f9ac815f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40526/0 (socket says 172.21.15.84:40526) 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 -- 172.21.15.84:0/4248201216 learned_addr learned my addr 172.21.15.84:0/4248201216 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acc99b700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80b83c0 0x7f9ac80c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9acd19c700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac8149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 -- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 msgr2=0x7f9ac8149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac8149d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 -- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80b83c0 msgr2=0x7f9ac80c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80b83c0 0x7f9ac80c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.823+0000 7f9ac7fff700 1 -- 172.21.15.84:0/4248201216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40094a0 con 0x7f9ac80cfa90 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.824+0000 7f9ac7fff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 0x7f9ac815f1f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400e9b0 tx=0x7f9ab400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.824+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400ee50 con 0x7f9ac80cfa90 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.824+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab400c850 con 0x7f9ac80cfa90 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.824+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400cc00 con 0x7f9ac80cfa90 2023-07-06T21:42:31.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.824+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac0009a70 con 0x7f9ac80cfa90 2023-07-06T21:42:31.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.824+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac815e9f0 con 0x7f9ac80cfa90 2023-07-06T21:42:31.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.825+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac815ac90 con 0x7f9ac80cfa90 2023-07-06T21:42:31.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.826+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ab4013470 con 0x7f9ac80cfa90 2023-07-06T21:42:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.828+0000 7f9ac57fa700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9aac06ace0 0x7f9aac06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.828+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ab40891b0 con 0x7f9ac80cfa90 2023-07-06T21:42:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.828+0000 7f9acc99b700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9aac06ace0 0x7f9aac06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.829+0000 7f9acc99b700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9aac06ace0 0x7f9aac06d190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000f60 tx=0x7f9abc007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:31.832+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ab4057c50 con 0x7f9ac80cfa90 2023-07-06T21:42:32.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.032+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ac815a9b0 con 0x7f9ac80cfa90 2023-07-06T21:42:32.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.035+0000 7f9ac57fa700 1 -- 172.21.15.84:0/4248201216 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7f9ab40577e0 con 0x7f9ac80cfa90 2023-07-06T21:42:32.038 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:32.038 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9aac06ace0 msgr2=0x7f9aac06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9aac06ace0 0x7f9aac06d190 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9abc000f60 tx=0x7f9abc007e70 comp rx=0 tx=0).stop 2023-07-06T21:42:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 msgr2=0x7f9ac815f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 0x7f9ac815f1f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400e9b0 tx=0x7f9ab400ecc0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 shutdown_connections 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9aac06ace0 0x7f9aac06d190 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ac80d3710 0x7f9ac8149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ac80b83c0 0x7f9ac80c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 --2- 172.21.15.84:0/4248201216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ac80cfa90 0x7f9ac815f1f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 >> 172.21.15.84:0/4248201216 conn(0x7f9ac806b350 msgr2=0x7f9ac80b6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:32.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 shutdown_connections 2023-07-06T21:42:32.042 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.037+0000 7f9acebff700 1 -- 172.21.15.84:0/4248201216 wait complete. 2023-07-06T21:42:32.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:42:08.774391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:42:09.558402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:32.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:32.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.254+0000 7f68709ff700 1 Processor -- start 2023-07-06T21:42:32.258 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.255+0000 7f68709ff700 1 -- start start 2023-07-06T21:42:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f68709ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b81c0 0x7f686c0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f68709ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f686c0b1a80 0x7f686c0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.259 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f68709ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f686c0ceda0 0x7f686c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f68709ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0d1830 con 0x7f686c0b1a80 2023-07-06T21:42:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f68709ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f686c0cf330 con 0x7f686c0ceda0 2023-07-06T21:42:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f68709ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0d1270 con 0x7f686c0b81c0 2023-07-06T21:42:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f686c0b1a80 0x7f686c0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f686c0b1a80 0x7f686c0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40530/0 (socket says 172.21.15.84:40530) 2023-07-06T21:42:32.260 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 -- 172.21.15.84:0/735367647 learned_addr learned my addr 172.21.15.84:0/735367647 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f686a59c700 1 --2- 172.21.15.84:0/735367647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b81c0 0x7f686c0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f686ad9d700 1 --2- 172.21.15.84:0/735367647 >> v2:172.21.15.107:3300/0 conn(0x7f686c0ceda0 0x7f686c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 -- 172.21.15.84:0/735367647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b81c0 msgr2=0x7f686c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 --2- 172.21.15.84:0/735367647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b81c0 0x7f686c0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 -- 172.21.15.84:0/735367647 >> v2:172.21.15.107:3300/0 conn(0x7f686c0ceda0 msgr2=0x7f686c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 --2- 172.21.15.84:0/735367647 >> v2:172.21.15.107:3300/0 conn(0x7f686c0ceda0 0x7f686c15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.261 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f6863fff700 1 -- 172.21.15.84:0/735367647 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f686c0d1550 con 0x7f686c0b1a80 2023-07-06T21:42:32.262 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.256+0000 7f686ad9d700 1 --2- 172.21.15.84:0/735367647 >> v2:172.21.15.107:3300/0 conn(0x7f686c0ceda0 0x7f686c15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f6863fff700 1 --2- 172.21.15.84:0/735367647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b1a80 0x7f686c0cfb40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f686c0b8d30 tx=0x7f6854009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9212e857e688e2d server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f6869d9b700 1 -- 172.21.15.84:0/735367647 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854011830 con 0x7f686c0b1a80 2023-07-06T21:42:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f6869d9b700 1 -- 172.21.15.84:0/735367647 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854011990 con 0x7f686c0b1a80 2023-07-06T21:42:32.263 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f6869d9b700 1 -- 172.21.15.84:0/735367647 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854011d20 con 0x7f686c0b1a80 2023-07-06T21:42:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 -- 172.21.15.84:0/735367647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b1a80 msgr2=0x7f686c0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 --2- 172.21.15.84:0/735367647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b1a80 0x7f686c0cfb40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f686c0b8d30 tx=0x7f6854009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 -- 172.21.15.84:0/735367647 shutdown_connections 2023-07-06T21:42:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 --2- 172.21.15.84:0/735367647 >> v2:172.21.15.107:3300/0 conn(0x7f686c0ceda0 0x7f686c15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 --2- 172.21.15.84:0/735367647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b81c0 0x7f686c0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.264 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 --2- 172.21.15.84:0/735367647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b1a80 0x7f686c0cfb40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.257+0000 7f68709ff700 1 -- 172.21.15.84:0/735367647 >> 172.21.15.84:0/735367647 conn(0x7f686c06b2b0 msgr2=0x7f686c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.258+0000 7f68709ff700 1 -- 172.21.15.84:0/735367647 shutdown_connections 2023-07-06T21:42:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.258+0000 7f68709ff700 1 -- 172.21.15.84:0/735367647 wait complete. 2023-07-06T21:42:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.259+0000 7f68709ff700 1 Processor -- start 2023-07-06T21:42:32.265 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 -- start start 2023-07-06T21:42:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b1a80 0x7f686c157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 0x7f686c0c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f686c0ceda0 0x7f686c0c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0c77d0 con 0x7f686c0b81c0 2023-07-06T21:42:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0c6c60 con 0x7f686c0ceda0 2023-07-06T21:42:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.260+0000 7f68709ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f686c0c6f30 con 0x7f686c0b1a80 2023-07-06T21:42:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 0x7f686c0c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 0x7f686c0c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40536/0 (socket says 172.21.15.84:40536) 2023-07-06T21:42:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 -- 172.21.15.84:0/3337192187 learned_addr learned my addr 172.21.15.84:0/3337192187 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 -- 172.21.15.84:0/3337192187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b1a80 msgr2=0x7f686c157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b1a80 0x7f686c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 -- 172.21.15.84:0/3337192187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f686c0ceda0 msgr2=0x7f686c0c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f686c0ceda0 0x7f686c0c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 -- 172.21.15.84:0/3337192187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854009550 con 0x7f686c0b81c0 2023-07-06T21:42:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6863fff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 0x7f686c0c7b90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6854000c00 tx=0x7f6854004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.261+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854004380 con 0x7f686c0b81c0 2023-07-06T21:42:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.262+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685401f070 con 0x7f686c0b81c0 2023-07-06T21:42:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.262+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c0c7210 con 0x7f686c0b81c0 2023-07-06T21:42:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.262+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540036a0 con 0x7f686c0b81c0 2023-07-06T21:42:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.262+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c1545f0 con 0x7f686c0b81c0 2023-07-06T21:42:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.263+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c1547e0 con 0x7f686c0b81c0 2023-07-06T21:42:32.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.264+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f6854021030 con 0x7f686c0b81c0 2023-07-06T21:42:32.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.264+0000 7f6862ffd700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f684c06ace0 0x7f684c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.265+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f685408a840 con 0x7f686c0b81c0 2023-07-06T21:42:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.265+0000 7f686a59c700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f684c06ace0 0x7f684c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.265+0000 7f686a59c700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f684c06ace0 0x7f684c06d190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f685c006b20 tx=0x7f685c006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:32.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.269+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f6854055980 con 0x7f686c0b81c0 2023-07-06T21:42:32.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.460+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f686c04e1d0 con 0x7f686c0b81c0 2023-07-06T21:42:32.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.461+0000 7f6862ffd700 1 -- 172.21.15.84:0/3337192187 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6854015020 con 0x7f686c0b81c0 2023-07-06T21:42:32.464 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f684c06ace0 msgr2=0x7f684c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f684c06ace0 0x7f684c06d190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f685c006b20 tx=0x7f685c006ab0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 msgr2=0x7f686c0c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 0x7f686c0c7b90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6854000c00 tx=0x7f6854004750 comp rx=0 tx=0).stop 2023-07-06T21:42:32.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 shutdown_connections 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f684c06ace0 0x7f684c06d190 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f686c0ceda0 0x7f686c0c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f686c0b1a80 0x7f686c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 --2- 172.21.15.84:0/3337192187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f686c0b81c0 0x7f686c0c7b90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 >> 172.21.15.84:0/3337192187 conn(0x7f686c06b2b0 msgr2=0x7f686c0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 shutdown_connections 2023-07-06T21:42:32.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.463+0000 7f68709ff700 1 -- 172.21.15.84:0/3337192187 wait complete. 2023-07-06T21:42:32.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.678+0000 7fe8e0dff700 1 Processor -- start 2023-07-06T21:42:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.679+0000 7fe8e0dff700 1 -- start start 2023-07-06T21:42:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8e0dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe8dc069e90 0x7fe8dc06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8e0dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8dc06dde0 0x7fe8dc06a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8e0dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8dc0c94b0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8e0dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc06d350 con 0x7fe8dc06dde0 2023-07-06T21:42:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8e0dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8dc06a7a0 con 0x7fe8dc069e90 2023-07-06T21:42:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8dc06dde0 0x7fe8dc06a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe8dc06dde0 0x7fe8dc06a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40550/0 (socket says 172.21.15.84:40550) 2023-07-06T21:42:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 -- 172.21.15.84:0/1761623302 learned_addr learned my addr 172.21.15.84:0/1761623302 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:32.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8da59c700 1 --2- 172.21.15.84:0/1761623302 >> v2:172.21.15.107:3300/0 conn(0x7fe8dc069e90 0x7fe8dc06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 -- 172.21.15.84:0/1761623302 >> v2:172.21.15.107:3300/0 conn(0x7fe8dc069e90 msgr2=0x7fe8dc06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:32.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 --2- 172.21.15.84:0/1761623302 >> v2:172.21.15.107:3300/0 conn(0x7fe8dc069e90 0x7fe8dc06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 -- 172.21.15.84:0/1761623302 >> v1:172.21.15.107:6789/0 conn(0x7fe8dc0cf280 legacy=0x7fe8dc0b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.680+0000 7fe8d3fff700 1 -- 172.21.15.84:0/1761623302 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8dc0ca9b0 con 0x7fe8dc06dde0 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.681+0000 7fe8d3fff700 1 --2- 172.21.15.84:0/1761623302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc06a260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06e830 tx=0x7fe8c4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57cba2929554d6 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.681+0000 7fe8d9d9b700 1 -- 172.21.15.84:0/1761623302 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4004230 con 0x7fe8dc06dde0 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.681+0000 7fe8d9d9b700 1 -- 172.21.15.84:0/1761623302 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8c4004390 con 0x7fe8dc06dde0 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.681+0000 7fe8d9d9b700 1 -- 172.21.15.84:0/1761623302 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40036a0 con 0x7fe8dc06dde0 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.681+0000 7fe8e0dff700 1 -- 172.21.15.84:0/1761623302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc06dde0 msgr2=0x7fe8dc06a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.686 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.681+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/1761623302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc06a260 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06e830 tx=0x7fe8c4009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.682+0000 7fe8e0dff700 1 -- 172.21.15.84:0/1761623302 shutdown_connections 2023-07-06T21:42:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.682+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/1761623302 >> v2:172.21.15.107:3300/0 conn(0x7fe8dc069e90 0x7fe8dc06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.682+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/1761623302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc06a260 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.682+0000 7fe8e0dff700 1 -- 172.21.15.84:0/1761623302 >> 172.21.15.84:0/1761623302 conn(0x7fe8dc0cbf10 msgr2=0x7fe8dc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.682+0000 7fe8e0dff700 1 -- 172.21.15.84:0/1761623302 shutdown_connections 2023-07-06T21:42:32.687 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.682+0000 7fe8e0dff700 1 -- 172.21.15.84:0/1761623302 wait complete. 2023-07-06T21:42:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.684+0000 7fe8e0dff700 1 Processor -- start 2023-07-06T21:42:32.688 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.684+0000 7fe8e0dff700 1 -- start start 2023-07-06T21:42:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.685+0000 7fe8e0dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8dc069e90 0x7fe8dc15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.685+0000 7fe8e0dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc15c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8e0dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 0x7fe8dc15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.689 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8e0dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c7a00 con 0x7fe8dc0cf280 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8e0dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c7ce0 con 0x7fe8dc06dde0 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8e0dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8dc0c7fc0 con 0x7fe8dc069e90 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 0x7fe8dc15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 0x7fe8dc15c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40552/0 (socket says 172.21.15.84:40552) 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 -- 172.21.15.84:0/2968530607 learned_addr learned my addr 172.21.15.84:0/2968530607 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8d3fff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.690 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8da59c700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8dc069e90 0x7fe8dc15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 -- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8dc069e90 msgr2=0x7fe8dc15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8dc069e90 0x7fe8dc15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 -- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8dc06dde0 msgr2=0x7fe8dc15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc15c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8dad9d700 1 -- 172.21.15.84:0/2968530607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c4009550 con 0x7fe8dc0cf280 2023-07-06T21:42:32.691 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.686+0000 7fe8da59c700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8dc069e90 0x7fe8dc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.687+0000 7fe8dad9d700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 0x7fe8dc15c650 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b970 tx=0x7fe8d400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:32.692 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.687+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400c9b0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.687+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400cb10 con 0x7fe8dc0cf280 2023-07-06T21:42:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.687+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8dc0c82a0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.687+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4014440 con 0x7fe8dc0cf280 2023-07-06T21:42:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.687+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8dc0c7770 con 0x7fe8dc0cf280 2023-07-06T21:42:32.693 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.688+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8dc0c8490 con 0x7fe8dc0cf280 2023-07-06T21:42:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.689+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe8d401d3b0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.694 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.690+0000 7fe8d2ffd700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe8bc06ace0 0x7fe8bc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.690+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe8d40890f0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.690+0000 7fe8da59c700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe8bc06ace0 0x7fe8bc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:32.695 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.690+0000 7fe8da59c700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe8bc06ace0 0x7fe8bc06d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06a6f0 tx=0x7fe8cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:32.697 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.694+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe8d4054190 con 0x7fe8dc0cf280 2023-07-06T21:42:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.891+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe8dc04e1d0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.891+0000 7fe8d2ffd700 1 -- 172.21.15.84:0/2968530607 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe8d40577b0 con 0x7fe8dc0cf280 2023-07-06T21:42:32.895 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe8bc06ace0 msgr2=0x7fe8bc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe8bc06ace0 0x7fe8bc06d190 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc06a6f0 tx=0x7fe8cc008040 comp rx=0 tx=0).stop 2023-07-06T21:42:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 msgr2=0x7fe8dc15c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 0x7fe8dc15c650 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400b970 tx=0x7fe8d400bd30 comp rx=0 tx=0).stop 2023-07-06T21:42:32.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 shutdown_connections 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe8bc06ace0 0x7fe8bc06d190 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe8dc0cf280 0x7fe8dc15c650 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8dc069e90 0x7fe8dc15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 --2- 172.21.15.84:0/2968530607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8dc06dde0 0x7fe8dc15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 >> 172.21.15.84:0/2968530607 conn(0x7fe8dc0cbf10 msgr2=0x7fe8dc0cc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 shutdown_connections 2023-07-06T21:42:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:32.894+0000 7fe8e0dff700 1 -- 172.21.15.84:0/2968530607 wait complete. 2023-07-06T21:42:32.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:42:33.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.110+0000 7f7aa5fff700 1 Processor -- start 2023-07-06T21:42:33.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.111+0000 7f7aa5fff700 1 -- start start 2023-07-06T21:42:33.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7aa5fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b1a80 0x7f7aa00d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7aa5fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa00d1830 con 0x7f7aa00b81c0 2023-07-06T21:42:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7aa5fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa00cf330 con 0x7f7aa00cfd10 2023-07-06T21:42:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7aa5fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa00d1270 con 0x7f7aa00b1a80 2023-07-06T21:42:33.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7a9f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7aa00b81c0 0x7f7aa00cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41028/0 (socket says 172.21.15.84:41028) 2023-07-06T21:42:33.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7a9f7fe700 1 -- 172.21.15.84:0/4234913932 learned_addr learned my addr 172.21.15.84:0/4234913932 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:33.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7a9ffff700 1 --2- 172.21.15.84:0/4234913932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b1a80 0x7f7aa00d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836436587 0 0) 0x7f7aa00d1830 con 0x7f7aa00b81c0 2023-07-06T21:42:33.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.112+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a880035b0 con 0x7f7aa00b81c0 2023-07-06T21:42:33.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77969456 0 0) 0x7f7aa00cf330 con 0x7f7aa00cfd10 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa00d1830 con 0x7f7aa00cfd10 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9ffff700 1 -- 172.21.15.84:0/4234913932 >> v1:172.21.15.107:6789/0 conn(0x7f7aa00cfd10 legacy=0x7f7aa00d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9ffff700 1 -- 172.21.15.84:0/4234913932 >> v1:172.21.15.84:6789/0 conn(0x7f7aa00b81c0 legacy=0x7f7aa00cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9ffff700 1 -- 172.21.15.84:0/4234913932 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa00d1550 con 0x7f7aa00b1a80 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9ffff700 1 --2- 172.21.15.84:0/4234913932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b1a80 0x7f7aa00d2a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009dc0 tx=0x7f7a9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e3ec1717b258ad server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800e070 con 0x7f7aa00b1a80 2023-07-06T21:42:33.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9800bb20 con 0x7f7aa00b1a80 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.113+0000 7f7a9e7fc700 1 -- 172.21.15.84:0/4234913932 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98012430 con 0x7f7aa00b1a80 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 -- 172.21.15.84:0/4234913932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b1a80 msgr2=0x7f7aa00d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/4234913932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b1a80 0x7f7aa00d2a90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009dc0 tx=0x7f7a9800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 -- 172.21.15.84:0/4234913932 shutdown_connections 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/4234913932 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b1a80 0x7f7aa00d2a90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 -- 172.21.15.84:0/4234913932 >> 172.21.15.84:0/4234913932 conn(0x7f7aa006b2b0 msgr2=0x7f7aa006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:33.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 -- 172.21.15.84:0/4234913932 shutdown_connections 2023-07-06T21:42:33.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.114+0000 7f7aa5fff700 1 -- 172.21.15.84:0/4234913932 wait complete. 2023-07-06T21:42:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.116+0000 7f7aa5fff700 1 Processor -- start 2023-07-06T21:42:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.116+0000 7f7aa5fff700 1 -- start start 2023-07-06T21:42:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7aa5fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7aa00b1a80 0x7f7aa00b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7aa5fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b81c0 0x7f7aa015ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7aa5fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 0x7f7aa015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7aa5fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa015ca20 con 0x7f7aa00cfd10 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7aa5fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa015ccc0 con 0x7f7aa00b1a80 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7aa5fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa015cfa0 con 0x7f7aa00b81c0 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 0x7f7aa015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 0x7f7aa015f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39470/0 (socket says 172.21.15.84:39470) 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 -- 172.21.15.84:0/2659526435 learned_addr learned my addr 172.21.15.84:0/2659526435 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:33.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9effd700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b81c0 0x7f7aa015ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9f7fe700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7aa00b1a80 0x7f7aa00b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 -- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b81c0 msgr2=0x7f7aa015ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b81c0 0x7f7aa015ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 -- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7aa00b1a80 msgr2=0x7f7aa00b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7aa00b1a80 0x7f7aa00b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.117+0000 7f7a9ffff700 1 -- 172.21.15.84:0/2659526435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98009a70 con 0x7f7aa00cfd10 2023-07-06T21:42:33.127 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.118+0000 7f7a9ffff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 0x7f7aa015f3a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7a98014040 tx=0x7f7a98004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.118+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9801b810 con 0x7f7aa00cfd10 2023-07-06T21:42:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.118+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a98020070 con 0x7f7aa00cfd10 2023-07-06T21:42:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.118+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa015d280 con 0x7f7aa00cfd10 2023-07-06T21:42:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.118+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9801bbd0 con 0x7f7aa00cfd10 2023-07-06T21:42:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.118+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa015d840 con 0x7f7aa00cfd10 2023-07-06T21:42:33.128 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.119+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa004e1d0 con 0x7f7aa00cfd10 2023-07-06T21:42:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.120+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a9801bd30 con 0x7f7aa00cfd10 2023-07-06T21:42:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.121+0000 7f7a87fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a8c06ace0 0x7f7a8c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.121+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a98022070 con 0x7f7aa00cfd10 2023-07-06T21:42:33.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.121+0000 7f7a9f7fe700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a8c06ace0 0x7f7a8c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.122+0000 7f7a9f7fe700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a8c06ace0 0x7f7a8c06d190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a90007fd0 tx=0x7f7a90007e00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.125+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7a98056480 con 0x7f7aa00cfd10 2023-07-06T21:42:33.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.324+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7aa00c9a80 con 0x7f7aa00cfd10 2023-07-06T21:42:33.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.325+0000 7f7a87fff700 1 -- 172.21.15.84:0/2659526435 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a98059aa0 con 0x7f7aa00cfd10 2023-07-06T21:42:33.328 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:42:33.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a8c06ace0 msgr2=0x7f7a8c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a8c06ace0 0x7f7a8c06d190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a90007fd0 tx=0x7f7a90007e00 comp rx=0 tx=0).stop 2023-07-06T21:42:33.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 msgr2=0x7f7aa015f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 0x7f7aa015f3a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7a98014040 tx=0x7f7a98004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 shutdown_connections 2023-07-06T21:42:33.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a8c06ace0 0x7f7a8c06d190 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7aa00cfd10 0x7f7aa015f3a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7aa00b1a80 0x7f7aa00b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 --2- 172.21.15.84:0/2659526435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7aa00b81c0 0x7f7aa015ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 >> 172.21.15.84:0/2659526435 conn(0x7f7aa006b2b0 msgr2=0x7f7aa00d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:33.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 shutdown_connections 2023-07-06T21:42:33.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.327+0000 7f7aa5fff700 1 -- 172.21.15.84:0/2659526435 wait complete. 2023-07-06T21:42:33.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:42:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.541+0000 7fe677dff700 1 Processor -- start 2023-07-06T21:42:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.541+0000 7fe677dff700 1 -- start start 2023-07-06T21:42:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.542+0000 7fe677dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700cfd10 0x7fe6700d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.542+0000 7fe677dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700d1830 con 0x7fe6700b1a80 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.542+0000 7fe677dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6700cf330 con 0x7fe6700b81c0 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.542+0000 7fe677dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700d1270 con 0x7fe6700cfd10 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe67639c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fe6700b1a80 0x7fe6700d28c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41042/0 (socket says 172.21.15.84:41042) 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe67639c700 1 -- 172.21.15.84:0/3197659136 learned_addr learned my addr 172.21.15.84:0/3197659136 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe67539a700 1 --2- 172.21.15.84:0/3197659136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700cfd10 0x7fe6700d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1734614559 0 0) 0x7fe6700d1830 con 0x7fe6700b1a80 2023-07-06T21:42:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe65c0035b0 con 0x7fe6700b1a80 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983818866 0 0) 0x7fe6700cf330 con 0x7fe6700b81c0 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6700d1830 con 0x7fe6700b81c0 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe67539a700 1 -- 172.21.15.84:0/3197659136 >> v1:172.21.15.107:6789/0 conn(0x7fe6700b81c0 legacy=0x7fe6700cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325778626 0 0) 0x7fe65c0035b0 con 0x7fe6700b1a80 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe67539a700 1 -- 172.21.15.84:0/3197659136 >> v1:172.21.15.84:6789/0 conn(0x7fe6700b1a80 legacy=0x7fe6700d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.543+0000 7fe67539a700 1 -- 172.21.15.84:0/3197659136 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6700d1550 con 0x7fe6700cfd10 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.544+0000 7fe67539a700 1 --2- 172.21.15.84:0/3197659136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700cfd10 0x7fe6700d2380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe6600098a0 tx=0x7fe660009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7c0dbcaae3be7f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.544+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6680019f0 con 0x7fe6700cfd10 2023-07-06T21:42:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.544+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe660004030 con 0x7fe6700cfd10 2023-07-06T21:42:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.544+0000 7fe674b99700 1 -- 172.21.15.84:0/3197659136 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6600043c0 con 0x7fe6700cfd10 2023-07-06T21:42:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.544+0000 7fe677dff700 1 -- 172.21.15.84:0/3197659136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700cfd10 msgr2=0x7fe6700d2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.544+0000 7fe677dff700 1 --2- 172.21.15.84:0/3197659136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700cfd10 0x7fe6700d2380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe6600098a0 tx=0x7fe660009bb0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.545+0000 7fe677dff700 1 -- 172.21.15.84:0/3197659136 shutdown_connections 2023-07-06T21:42:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.545+0000 7fe677dff700 1 --2- 172.21.15.84:0/3197659136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700cfd10 0x7fe6700d2380 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.545+0000 7fe677dff700 1 -- 172.21.15.84:0/3197659136 >> 172.21.15.84:0/3197659136 conn(0x7fe67006b2b0 msgr2=0x7fe67006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.545+0000 7fe677dff700 1 -- 172.21.15.84:0/3197659136 shutdown_connections 2023-07-06T21:42:33.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.545+0000 7fe677dff700 1 -- 172.21.15.84:0/3197659136 wait complete. 2023-07-06T21:42:33.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.546+0000 7fe677dff700 1 Processor -- start 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 -- start start 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 0x7fe670156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700b81c0 0x7fe67015f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6700cfd10 0x7fe67015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe670159ca0 con 0x7fe6700b1a80 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700c9660 con 0x7fe6700cfd10 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.547+0000 7fe677dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6700c8a50 con 0x7fe6700b81c0 2023-07-06T21:42:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 0x7fe670156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 0x7fe670156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39476/0 (socket says 172.21.15.84:39476) 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 -- 172.21.15.84:0/367473875 learned_addr learned my addr 172.21.15.84:0/367473875 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe67539a700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700b81c0 0x7fe67015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe67639c700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6700cfd10 0x7fe67015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 -- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700b81c0 msgr2=0x7fe67015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700b81c0 0x7fe67015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 -- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6700cfd10 msgr2=0x7fe67015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6700cfd10 0x7fe67015f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 -- 172.21.15.84:0/367473875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6600094a0 con 0x7fe6700b1a80 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.548+0000 7fe675b9b700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 0x7fe670156330 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c550 tx=0x7fe66c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.549+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0052d0 con 0x7fe6700b1a80 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.549+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66c005430 con 0x7fe6700b1a80 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.549+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6700c8d30 con 0x7fe6700b1a80 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.549+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66c0038c0 con 0x7fe6700b1a80 2023-07-06T21:42:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.549+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6700c92f0 con 0x7fe6700b1a80 2023-07-06T21:42:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.550+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6700ce370 con 0x7fe6700b1a80 2023-07-06T21:42:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.551+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe66c003a20 con 0x7fe6700b1a80 2023-07-06T21:42:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.552+0000 7fe6667fc700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe65406ace0 0x7fe65406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.552+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe66c0888d0 con 0x7fe6700b1a80 2023-07-06T21:42:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.552+0000 7fe67539a700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe65406ace0 0x7fe65406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.552+0000 7fe67539a700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe65406ace0 0x7fe65406d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe6700d0100 tx=0x7fe660008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.556+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe66c053970 con 0x7fe6700b1a80 2023-07-06T21:42:33.756 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.753+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe67004e1d0 con 0x7fe6700b1a80 2023-07-06T21:42:33.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.753+0000 7fe6667fc700 1 -- 172.21.15.84:0/367473875 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe66c017020 con 0x7fe6700b1a80 2023-07-06T21:42:33.757 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:42:33.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe65406ace0 msgr2=0x7fe65406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe65406ace0 0x7fe65406d190 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe6700d0100 tx=0x7fe660008030 comp rx=0 tx=0).stop 2023-07-06T21:42:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 msgr2=0x7fe670156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 0x7fe670156330 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe66c00c550 tx=0x7fe66c00c860 comp rx=0 tx=0).stop 2023-07-06T21:42:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 shutdown_connections 2023-07-06T21:42:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe65406ace0 0x7fe65406d190 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6700cfd10 0x7fe67015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6700b1a80 0x7fe670156330 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 --2- 172.21.15.84:0/367473875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700b81c0 0x7fe67015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 >> 172.21.15.84:0/367473875 conn(0x7fe67006b2b0 msgr2=0x7fe6700b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.755+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 shutdown_connections 2023-07-06T21:42:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.756+0000 7fe677dff700 1 -- 172.21.15.84:0/367473875 wait complete. 2023-07-06T21:42:33.776 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:42:33.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.967+0000 7f5191fff700 1 Processor -- start 2023-07-06T21:42:33.970 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.967+0000 7f5191fff700 1 -- start start 2023-07-06T21:42:33.971 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.968+0000 7f5191fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f518c0b83c0 0x7f518c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.968+0000 7f5191fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0d3710 0x7f518c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.968+0000 7f5191fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c0cfee0 con 0x7f518c0cfa90 2023-07-06T21:42:33.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.968+0000 7f5191fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c0d0a80 con 0x7f518c0b83c0 2023-07-06T21:42:33.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.968+0000 7f5191fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0b8790 con 0x7f518c0d3710 2023-07-06T21:42:33.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.968+0000 7f518b7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f518c0b83c0 0x7f518c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f518c0cfa90 0x7f518c15feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41044/0 (socket says 172.21.15.84:41044) 2023-07-06T21:42:33.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518bfff700 1 -- 172.21.15.84:0/4059861082 learned_addr learned my addr 172.21.15.84:0/4059861082 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:33.973 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518affd700 1 --2- 172.21.15.84:0/4059861082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0d3710 0x7f518c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518a7fc700 1 -- 172.21.15.84:0/4059861082 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2746311538 0 0) 0x7f5184001490 con 0x7f518c0cfa90 2023-07-06T21:42:33.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518a7fc700 1 -- 172.21.15.84:0/4059861082 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51740035b0 con 0x7f518c0cfa90 2023-07-06T21:42:33.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518b7fe700 1 -- 172.21.15.84:0/4059861082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0d3710 msgr2=0x7f518c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:33.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518b7fe700 1 --2- 172.21.15.84:0/4059861082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0d3710 0x7f518c0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518b7fe700 1 -- 172.21.15.84:0/4059861082 >> v1:172.21.15.84:6789/0 conn(0x7f518c0cfa90 legacy=0x7f518c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518b7fe700 1 -- 172.21.15.84:0/4059861082 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0b4e70 con 0x7f518c0b83c0 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518b7fe700 1 --2- 172.21.15.84:0/4059861082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c0cd200 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f517c009840 tx=0x7f517c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a0a0d268bbf439 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.969+0000 7f518a7fc700 1 -- 172.21.15.84:0/4059861082 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c011830 con 0x7f518c0b83c0 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f518a7fc700 1 -- 172.21.15.84:0/4059861082 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c011990 con 0x7f518c0b83c0 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f518a7fc700 1 -- 172.21.15.84:0/4059861082 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c011d20 con 0x7f518c0b83c0 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 -- 172.21.15.84:0/4059861082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 msgr2=0x7f518c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 --2- 172.21.15.84:0/4059861082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c0cd200 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f517c009840 tx=0x7f517c009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 -- 172.21.15.84:0/4059861082 shutdown_connections 2023-07-06T21:42:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 --2- 172.21.15.84:0/4059861082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c0cd200 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 --2- 172.21.15.84:0/4059861082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0d3710 0x7f518c0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 -- 172.21.15.84:0/4059861082 >> 172.21.15.84:0/4059861082 conn(0x7f518c06b350 msgr2=0x7f518c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 -- 172.21.15.84:0/4059861082 shutdown_connections 2023-07-06T21:42:33.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.970+0000 7f5191fff700 1 -- 172.21.15.84:0/4059861082 wait complete. 2023-07-06T21:42:33.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.972+0000 7f5191fff700 1 Processor -- start 2023-07-06T21:42:33.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.972+0000 7f5191fff700 1 -- start start 2023-07-06T21:42:33.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f5191fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f5191fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0cfa90 0x7f518c0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f5191fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518c0d3710 0x7f518c0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f5191fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15ea70 con 0x7f518c0b83c0 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f5191fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0c9550 con 0x7f518c0d3710 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f5191fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c1512b0 con 0x7f518c0cfa90 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f518b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f518b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39496/0 (socket says 172.21.15.84:39496) 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f518b7fe700 1 -- 172.21.15.84:0/4051965863 learned_addr learned my addr 172.21.15.84:0/4051965863 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.973+0000 7f518affd700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0cfa90 0x7f518c0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518bfff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518c0d3710 0x7f518c0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518b7fe700 1 -- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0cfa90 msgr2=0x7f518c0b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518b7fe700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0cfa90 0x7f518c0b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518b7fe700 1 -- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518c0d3710 msgr2=0x7f518c0b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518b7fe700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518c0d3710 0x7f518c0b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518b7fe700 1 -- 172.21.15.84:0/4051965863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0094a0 con 0x7f518c0b83c0 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518affd700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0cfa90 0x7f518c0b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:33.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.974+0000 7f518b7fe700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c150d70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f517c009fd0 tx=0x7f517c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.975+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0036a0 con 0x7f518c0b83c0 2023-07-06T21:42:33.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.975+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c01e070 con 0x7f518c0b83c0 2023-07-06T21:42:33.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.975+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c15e0d0 con 0x7f518c0b83c0 2023-07-06T21:42:33.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.975+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c003a60 con 0x7f518c0b83c0 2023-07-06T21:42:33.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.975+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c15e690 con 0x7f518c0b83c0 2023-07-06T21:42:33.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.976+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c0b4a10 con 0x7f518c0b83c0 2023-07-06T21:42:33.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.977+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f517c003bc0 con 0x7f518c0b83c0 2023-07-06T21:42:33.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.978+0000 7f5173fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f517806ace0 0x7f517806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:33.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.978+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f517c08a3d0 con 0x7f518c0b83c0 2023-07-06T21:42:33.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.978+0000 7f518affd700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f517806ace0 0x7f517806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:33.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.978+0000 7f518affd700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f517806ace0 0x7f517806d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f518c0d3b20 tx=0x7f5180008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:33.985 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:33.982+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f517c055470 con 0x7f518c0b83c0 2023-07-06T21:42:34.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.199+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f518c160810 con 0x7f518c0b83c0 2023-07-06T21:42:34.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.199+0000 7f5173fff700 1 -- 172.21.15.84:0/4051965863 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7f517c058a90 con 0x7f518c0b83c0 2023-07-06T21:42:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f517806ace0 msgr2=0x7f517806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f517806ace0 0x7f517806d190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f518c0d3b20 tx=0x7f5180008040 comp rx=0 tx=0).stop 2023-07-06T21:42:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 msgr2=0x7f518c150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c150d70 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f517c009fd0 tx=0x7f517c004750 comp rx=0 tx=0).stop 2023-07-06T21:42:34.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 shutdown_connections 2023-07-06T21:42:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f517806ace0 0x7f517806d190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f518c0d3710 0x7f518c0b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:34.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f518c0b83c0 0x7f518c150d70 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 --2- 172.21.15.84:0/4051965863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f518c0cfa90 0x7f518c0b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.201+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 >> 172.21.15.84:0/4051965863 conn(0x7f518c06b350 msgr2=0x7f518c0d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.202+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 shutdown_connections 2023-07-06T21:42:34.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:34.202+0000 7f5191fff700 1 -- 172.21.15.84:0/4051965863 wait complete. 2023-07-06T21:42:34.209 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:42:34.223 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:42:34.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:34.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:34.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[154913]: fuse finished with error 0 and tester_r 0 2023-07-06T21:42:34.292 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:42:40.394 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:42:40.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:40.395 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:42:40.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-07-06T21:42:40.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.638+0000 7f7a7d1ff700 1 Processor -- start 2023-07-06T21:42:40.642 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.639+0000 7f7a7d1ff700 1 -- start start 2023-07-06T21:42:40.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a7d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a780cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:40.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a7d1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a78160960 con 0x7f7a7806ba90 2023-07-06T21:42:40.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a7d1ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a7806bf00 con 0x7f7a780cd720 2023-07-06T21:42:40.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a7d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7806a5b0 con 0x7f7a780b1260 2023-07-06T21:42:40.643 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a7659c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7a7806ba90 0x7f7a780b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:41046/0 (socket says 172.21.15.84:41046) 2023-07-06T21:42:40.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a7659c700 1 -- 172.21.15.84:0/2382369223 learned_addr learned my addr 172.21.15.84:0/2382369223 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:40.644 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a76d9d700 1 --2- 172.21.15.84:0/2382369223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a780cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:40.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604076232 0 0) 0x7f7a78160960 con 0x7f7a7806ba90 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.640+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a600035b0 con 0x7f7a7806ba90 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512952426 0 0) 0x7f7a7806bf00 con 0x7f7a780cd720 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a78160960 con 0x7f7a780cd720 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a76d9d700 1 -- 172.21.15.84:0/2382369223 >> v1:172.21.15.107:6789/0 conn(0x7f7a780cd720 legacy=0x7f7a780cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a76d9d700 1 -- 172.21.15.84:0/2382369223 >> v1:172.21.15.84:6789/0 conn(0x7f7a7806ba90 legacy=0x7f7a780b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663354218 0 0) 0x7f7a600035b0 con 0x7f7a7806ba90 2023-07-06T21:42:40.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a76d9d700 1 -- 172.21.15.84:0/2382369223 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7815c820 con 0x7f7a780b1260 2023-07-06T21:42:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a76d9d700 1 --2- 172.21.15.84:0/2382369223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a780cef60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56205c4f7498bc6c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c001670 con 0x7f7a780b1260 2023-07-06T21:42:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68004030 con 0x7f7a780b1260 2023-07-06T21:42:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.641+0000 7f7a75d9b700 1 -- 172.21.15.84:0/2382369223 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680043c0 con 0x7f7a780b1260 2023-07-06T21:42:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/2382369223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 msgr2=0x7f7a780cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/2382369223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a780cef60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009870 tx=0x7f7a68009b80 comp rx=0 tx=0).stop 2023-07-06T21:42:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/2382369223 shutdown_connections 2023-07-06T21:42:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/2382369223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a780cef60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/2382369223 >> 172.21.15.84:0/2382369223 conn(0x7f7a780cc260 msgr2=0x7f7a780cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:40.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/2382369223 shutdown_connections 2023-07-06T21:42:40.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.642+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/2382369223 wait complete. 2023-07-06T21:42:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.644+0000 7f7a7d1ff700 1 Processor -- start 2023-07-06T21:42:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.645+0000 7f7a7d1ff700 1 -- start start 2023-07-06T21:42:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7d1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a7806ba90 0x7f7a78145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7d1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a781567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7d1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 0x7f7a78156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:40.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7d1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815db90 con 0x7f7a780cd720 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7d1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7815e430 con 0x7f7a7806ba90 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7d1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a781458d0 con 0x7f7a780b1260 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7759e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 0x7f7a78156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7759e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 0x7f7a78156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39502/0 (socket says 172.21.15.84:39502) 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7759e700 1 -- 172.21.15.84:0/227233288 learned_addr learned my addr 172.21.15.84:0/227233288 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a7659c700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a781567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:40.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.646+0000 7f7a76d9d700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a7806ba90 0x7f7a78145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7759e700 1 -- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 msgr2=0x7f7a781567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7759e700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a781567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7759e700 1 -- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a7806ba90 msgr2=0x7f7a78145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7759e700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a7806ba90 0x7f7a78145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7759e700 1 -- 172.21.15.84:0/227233288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680094a0 con 0x7f7a780cd720 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a76d9d700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a7806ba90 0x7f7a78145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:40.653 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7759e700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 0x7f7a78156d20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000cc20 tx=0x7f7a7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a70003c00 con 0x7f7a780cd720 2023-07-06T21:42:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a70003d60 con 0x7f7a780cd720 2023-07-06T21:42:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7000abe0 con 0x7f7a780cd720 2023-07-06T21:42:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a781460d0 con 0x7f7a780cd720 2023-07-06T21:42:40.654 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.647+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7815dde0 con 0x7f7a780cd720 2023-07-06T21:42:40.655 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.648+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7815e570 con 0x7f7a780cd720 2023-07-06T21:42:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.649+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7a7000ad40 con 0x7f7a780cd720 2023-07-06T21:42:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.651+0000 7f7a5f7fe700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a6406ace0 0x7f7a6406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.651+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7a70089630 con 0x7f7a780cd720 2023-07-06T21:42:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.651+0000 7f7a76d9d700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a6406ace0 0x7f7a6406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:40.656 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.651+0000 7f7a76d9d700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a6406ace0 0x7f7a6406d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a680090b0 tx=0x7f7a68008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:40.658 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.655+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7a700546d0 con 0x7f7a780cd720 2023-07-06T21:42:40.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.895+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7a78065f40 con 0x7f7a780cd720 2023-07-06T21:42:40.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.895+0000 7f7a5f7fe700 1 -- 172.21.15.84:0/227233288 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7a70057cf0 con 0x7f7a780cd720 2023-07-06T21:42:40.899 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-06T21:42:40.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a6406ace0 msgr2=0x7f7a6406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a6406ace0 0x7f7a6406d190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a680090b0 tx=0x7f7a68008030 comp rx=0 tx=0).stop 2023-07-06T21:42:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 msgr2=0x7f7a78156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 0x7f7a78156d20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7a7000cc20 tx=0x7f7a7000a060 comp rx=0 tx=0).stop 2023-07-06T21:42:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 shutdown_connections 2023-07-06T21:42:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7a6406ace0 0x7f7a6406d190 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a780cd720 0x7f7a78156d20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a7806ba90 0x7f7a78145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 --2- 172.21.15.84:0/227233288 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a780b1260 0x7f7a781567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 >> 172.21.15.84:0/227233288 conn(0x7f7a780cc260 msgr2=0x7f7a780cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 shutdown_connections 2023-07-06T21:42:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:40.898+0000 7f7a7d1ff700 1 -- 172.21.15.84:0/227233288 wait complete. 2023-07-06T21:42:40.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-06T21:42:40.920 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-06T21:42:40.920 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2023-07-06T21:42:40.920 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-06T21:42:40.920 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:40.921 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-06T21:42:40.921 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-06T21:42:40.921 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-06T21:42:40.921 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:40.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:40.922 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2023-07-06T21:42:40.965 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-06T21:42:40.965 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-06T21:42:40.965 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2023-07-06T21:42:40.965 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.965 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:40.966 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2023-07-06T21:42:40.967 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-06T21:42:40.967 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:40.967 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-06T21:42:40.967 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:40.967 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-07-06T21:42:40.967 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 42041sec preferred_lft 42041sec 2023-07-06T21:42:40.968 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::5e4d:c153:3a7b:e6e2/64 scope link noprefixroute 2023-07-06T21:42:40.968 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.968 INFO:teuthology.orchestra.run.smithi084.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-07-06T21:42:40.968 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff 2023-07-06T21:42:40.969 INFO:teuthology.orchestra.run.smithi084.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-07-06T21:42:40.969 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.969 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::606a:65ff:fe8a:8858/64 scope link 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 66:70:c9:ca:f1:11 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::6470:c9ff:feca:f111/64 scope link 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 46:92:bb:16:ee:bb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:42:40.970 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::4492:bbff:fe16:eebb/64 scope link 2023-07-06T21:42:40.971 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2023-07-06T21:42:40.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:40.972 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-06T21:42:41.079 INFO:teuthology.orchestra.run.smithi084.stdout:1 2023-07-06T21:42:41.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:41.083 DEBUG:teuthology.orchestra.run.smithi084:> route 2023-07-06T21:42:41.128 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2023-07-06T21:42:41.128 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-06T21:42:41.128 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-07-06T21:42:41.128 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-07-06T21:42:41.128 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-06T21:42:41.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:41.130 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec bash -c ' 2023-07-06T21:42:41.130 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2023-07-06T21:42:41.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-07-06T21:42:41.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-07-06T21:42:41.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-07-06T21:42:41.131 DEBUG:teuthology.orchestra.run.smithi084:> ') 2023-07-06T21:42:41.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:41.336 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:42:41.379 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:42:41.379 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:42:41.380 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:41.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:41.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2023-07-06T21:42:41.487 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-07-06T21:42:41.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:42:41.488 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:41.530 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-07-06T21:42:41.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-06T21:42:41.531 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:41.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:42:41.718 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:42:41.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:42:41.793 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:42:41.793 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:42:41.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:41.794 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:42:41.836 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-06T21:42:41.837 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-07-06T21:42:41.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo modprobe fuse 2023-07-06T21:42:41.951 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-06T21:42:41.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-06T21:42:42.023 INFO:teuthology.orchestra.run.smithi084.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-06T21:42:42.027 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-06T21:42:42.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:42.028 DEBUG:teuthology.orchestra.run.smithi084:> ls /sys/fs/fuse/connections 2023-07-06T21:42:42.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:2023-07-06T21:42:42.051+0000 7f2bcb3c05c0 -1 init, newargv = 0x5582c0a96280 newargc=15 2023-07-06T21:42:42.054 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[156108]: starting ceph client 2023-07-06T21:42:42.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[156108]: starting fuse 2023-07-06T21:42:42.074 INFO:teuthology.orchestra.run.smithi084.stdout:42 2023-07-06T21:42:42.074 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-07-06T21:42:42.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:42:42.075 DEBUG:teuthology.orchestra.run.smithi084:> import glob 2023-07-06T21:42:42.075 DEBUG:teuthology.orchestra.run.smithi084:> import re 2023-07-06T21:42:42.076 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:42:42.076 DEBUG:teuthology.orchestra.run.smithi084:> import subprocess 2023-07-06T21:42:42.076 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:42.076 DEBUG:teuthology.orchestra.run.smithi084:> def _find_admin_socket(client_name): 2023-07-06T21:42:42.077 DEBUG:teuthology.orchestra.run.smithi084:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-06T21:42:42.077 DEBUG:teuthology.orchestra.run.smithi084:> files = glob.glob(asok_path) 2023-07-06T21:42:42.077 DEBUG:teuthology.orchestra.run.smithi084:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-07-06T21:42:42.078 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:42.078 DEBUG:teuthology.orchestra.run.smithi084:> # Given a non-glob path, it better be there 2023-07-06T21:42:42.079 DEBUG:teuthology.orchestra.run.smithi084:> if "*" not in asok_path: 2023-07-06T21:42:42.079 DEBUG:teuthology.orchestra.run.smithi084:> assert(len(files) == 1) 2023-07-06T21:42:42.081 DEBUG:teuthology.orchestra.run.smithi084:> return files[0] 2023-07-06T21:42:42.082 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:42.082 DEBUG:teuthology.orchestra.run.smithi084:> for f in files: 2023-07-06T21:42:42.083 DEBUG:teuthology.orchestra.run.smithi084:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-06T21:42:42.083 DEBUG:teuthology.orchestra.run.smithi084:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-06T21:42:42.083 DEBUG:teuthology.orchestra.run.smithi084:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-06T21:42:42.084 DEBUG:teuthology.orchestra.run.smithi084:> contents = proc_f.read() 2023-07-06T21:42:42.084 DEBUG:teuthology.orchestra.run.smithi084:> if mountpoint in contents: 2023-07-06T21:42:42.085 DEBUG:teuthology.orchestra.run.smithi084:> return f 2023-07-06T21:42:42.086 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-06T21:42:42.086 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:42.087 DEBUG:teuthology.orchestra.run.smithi084:> print(_find_admin_socket("client.0")) 2023-07-06T21:42:42.088 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:42:42.256 INFO:teuthology.orchestra.run.smithi084.stdout:/var/run/ceph/ceph-client.0.156108.asok 2023-07-06T21:42:42.435 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.156108.asok 2023-07-06T21:42:42.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:42.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.156108.asok status 2023-07-06T21:42:42.660 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:42:42.660 INFO:teuthology.orchestra.run.smithi084.stdout: "metadata": { 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_sha1": "0a781ef0807ae9473fb1306ef07036226ca4f1a6", 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 18.0.0-4488-g0a781ef0 (0a781ef0807ae9473fb1306ef07036226ca4f1a6) reef (dev)", 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "entity_id": "0", 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": "smithi084", 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "pid": "156108", 2023-07-06T21:42:42.661 INFO:teuthology.orchestra.run.smithi084.stdout: "root": "/" 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_count": 0, 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: "dentry_pinned_count": 0, 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 6429, 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: "inst": { 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: "name": { 2023-07-06T21:42:42.662 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "client", 2023-07-06T21:42:42.663 INFO:teuthology.orchestra.run.smithi084.stdout: "num": 6429 2023-07-06T21:42:42.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:42.663 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2467170483 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: } 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": { 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2023-07-06T21:42:42.664 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "192.168.0.1:0", 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2467170483 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: "inst_str": "client.6429 192.168.0.1:0/2467170483", 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: "addr_str": "192.168.0.1:0/2467170483", 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: "inode_count": 1, 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_epoch": 24, 2023-07-06T21:42:42.665 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch": 53, 2023-07-06T21:42:42.666 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_epoch_barrier": 0, 2023-07-06T21:42:42.666 INFO:teuthology.orchestra.run.smithi084.stdout: "blocklisted": false, 2023-07-06T21:42:42.666 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs" 2023-07-06T21:42:42.666 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:42:42.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:42.675 DEBUG:teuthology.orchestra.run.smithi084:> stat --file-system '--printf=%T 2023-07-06T21:42:42.675 DEBUG:teuthology.orchestra.run.smithi084:> ' -- /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:42.721 INFO:teuthology.orchestra.run.smithi084.stdout:fuseblk 2023-07-06T21:42:42.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:42.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:42.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:42.828 DEBUG:teuthology.orchestra.run.smithi084:> adjust-ulimits daemon-helper kill python3 -c ' 2023-07-06T21:42:42.829 DEBUG:teuthology.orchestra.run.smithi084:> import os 2023-07-06T21:42:42.829 DEBUG:teuthology.orchestra.run.smithi084:> import errno 2023-07-06T21:42:42.829 DEBUG:teuthology.orchestra.run.smithi084:> 2023-07-06T21:42:42.830 DEBUG:teuthology.orchestra.run.smithi084:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2023-07-06T21:42:42.830 DEBUG:teuthology.orchestra.run.smithi084:> try: 2023-07-06T21:42:42.830 DEBUG:teuthology.orchestra.run.smithi084:> if True: 2023-07-06T21:42:42.830 DEBUG:teuthology.orchestra.run.smithi084:> ret = os.read(fd, 1024) 2023-07-06T21:42:42.831 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-07-06T21:42:42.831 DEBUG:teuthology.orchestra.run.smithi084:> os.write(fd, b'"'"'content'"'"') 2023-07-06T21:42:42.831 DEBUG:teuthology.orchestra.run.smithi084:> except OSError as e: 2023-07-06T21:42:42.832 DEBUG:teuthology.orchestra.run.smithi084:> if e.errno != errno.EPERM: 2023-07-06T21:42:42.832 DEBUG:teuthology.orchestra.run.smithi084:> raise 2023-07-06T21:42:42.832 DEBUG:teuthology.orchestra.run.smithi084:> else: 2023-07-06T21:42:42.833 DEBUG:teuthology.orchestra.run.smithi084:> raise RuntimeError("client does not check permission of data pool") 2023-07-06T21:42:42.833 DEBUG:teuthology.orchestra.run.smithi084:> ' 2023-07-06T21:42:42.946 INFO:teuthology.orchestra.run.smithi084.stderr:Traceback (most recent call last): 2023-07-06T21:42:42.946 INFO:teuthology.orchestra.run.smithi084.stderr: File "", line 15, in 2023-07-06T21:42:42.946 INFO:teuthology.orchestra.run.smithi084.stderr:RuntimeError: client does not check permission of data pool 2023-07-06T21:42:43.120 INFO:teuthology.orchestra.run.smithi084.stderr:daemon-helper: command failed with exit status 1 2023-07-06T21:42:43.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:42:43.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2023-07-06T21:42:43.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.365+0000 7f03d91ff700 1 Processor -- start 2023-07-06T21:42:43.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.365+0000 7f03d91ff700 1 -- start start 2023-07-06T21:42:43.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.366+0000 7f03d91ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03d40703a0 0x7f03d4070770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:43.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.366+0000 7f03d91ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c8d50 0x7f03d40c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:43.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.366+0000 7f03d91ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d415cd30 con 0x7f03d40c97f0 2023-07-06T21:42:43.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.366+0000 7f03d91ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d4070cb0 con 0x7f03d40703a0 2023-07-06T21:42:43.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.366+0000 7f03d91ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d415cf80 con 0x7f03d40c8d50 2023-07-06T21:42:43.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03d40703a0 0x7f03d4070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:43.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f03d40703a0 0x7f03d4070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60976/0 (socket says 172.21.15.84:60976) 2023-07-06T21:42:43.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 -- 172.21.15.84:0/519549060 learned_addr learned my addr 172.21.15.84:0/519549060 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:43.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d259c700 1 --2- 172.21.15.84:0/519549060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c8d50 0x7f03d40c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:43.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 -- 172.21.15.84:0/519549060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c8d50 msgr2=0x7f03d40c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:43.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 --2- 172.21.15.84:0/519549060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c8d50 0x7f03d40c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 -- 172.21.15.84:0/519549060 >> v1:172.21.15.107:6789/0 conn(0x7f03d40c97f0 legacy=0x7f03d40c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 -- 172.21.15.84:0/519549060 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d415d260 con 0x7f03d40703a0 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.367+0000 7f03d2d9d700 1 --2- 172.21.15.84:0/519549060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40703a0 0x7f03d4070770 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f03c40098f0 tx=0x7f03c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f4a0d65b070b63 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d1d9b700 1 -- 172.21.15.84:0/519549060 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400b8a0 con 0x7f03d40703a0 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d1d9b700 1 -- 172.21.15.84:0/519549060 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c400ba00 con 0x7f03d40703a0 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d1d9b700 1 -- 172.21.15.84:0/519549060 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c400bd90 con 0x7f03d40703a0 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 -- 172.21.15.84:0/519549060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40703a0 msgr2=0x7f03d4070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:43.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 --2- 172.21.15.84:0/519549060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40703a0 0x7f03d4070770 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f03c40098f0 tx=0x7f03c4009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:43.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 -- 172.21.15.84:0/519549060 shutdown_connections 2023-07-06T21:42:43.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 --2- 172.21.15.84:0/519549060 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40703a0 0x7f03d4070770 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 --2- 172.21.15.84:0/519549060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c8d50 0x7f03d40c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 -- 172.21.15.84:0/519549060 >> 172.21.15.84:0/519549060 conn(0x7f03d40cbe10 msgr2=0x7f03d40ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:43.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 -- 172.21.15.84:0/519549060 shutdown_connections 2023-07-06T21:42:43.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.368+0000 7f03d91ff700 1 -- 172.21.15.84:0/519549060 wait complete. 2023-07-06T21:42:43.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.370+0000 7f03d91ff700 1 Processor -- start 2023-07-06T21:42:43.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.370+0000 7f03d91ff700 1 -- start start 2023-07-06T21:42:43.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d91ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d40703a0 0x7f03d414cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:43.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d91ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 0x7f03d40c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d91ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c97f0 0x7f03d40b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d91ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d414d1a0 con 0x7f03d40c8d50 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d91ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c6480 con 0x7f03d40703a0 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d91ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c6e50 con 0x7f03d40c97f0 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 0x7f03d40c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 0x7f03d40c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60984/0 (socket says 172.21.15.84:60984) 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.371+0000 7f03d259c700 1 -- 172.21.15.84:0/165310616 learned_addr learned my addr 172.21.15.84:0/165310616 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:43.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d2d9d700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d40703a0 0x7f03d414cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:43.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d359e700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c97f0 0x7f03d40b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:43.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d259c700 1 -- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c97f0 msgr2=0x7f03d40b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:43.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d259c700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c97f0 0x7f03d40b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d259c700 1 -- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d40703a0 msgr2=0x7f03d414cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d259c700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d40703a0 0x7f03d414cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d259c700 1 -- 172.21.15.84:0/165310616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4009550 con 0x7f03d40c8d50 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d259c700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 0x7f03d40c6910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d830 tx=0x7f03c800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c80125f0 con 0x7f03d40c8d50 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8012750 con 0x7f03d40c8d50 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8012b00 con 0x7f03d40c8d50 2023-07-06T21:42:43.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.372+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d40c5870 con 0x7f03d40c8d50 2023-07-06T21:42:43.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.373+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d40c5e30 con 0x7f03d40c8d50 2023-07-06T21:42:43.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.373+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d40c6630 con 0x7f03d40c8d50 2023-07-06T21:42:43.381 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.374+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f03c8012c60 con 0x7f03d40c8d50 2023-07-06T21:42:43.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.375+0000 7f03bb7fe700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f03c006ace0 0x7f03c006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:43.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.375+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03c8089530 con 0x7f03d40c8d50 2023-07-06T21:42:43.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.375+0000 7f03d2d9d700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f03c006ace0 0x7f03c006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:43.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.376+0000 7f03d2d9d700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f03c006ace0 0x7f03c006d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000c00 tx=0x7f03c4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:43.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.379+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f03d40c6630 con 0x7f03d40c8d50 2023-07-06T21:42:43.657 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.654+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f03d40c65f0 con 0x7f03d40c8d50 2023-07-06T21:42:43.661 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.658+0000 7f03bb7fe700 1 -- 172.21.15.84:0/165310616 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7f03c80545e0 con 0x7f03d40c8d50 2023-07-06T21:42:43.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f03c006ace0 msgr2=0x7f03c006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:43.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f03c006ace0 0x7f03c006d190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000c00 tx=0x7f03c4008030 comp rx=0 tx=0).stop 2023-07-06T21:42:43.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 msgr2=0x7f03d40c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:43.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 0x7f03d40c6910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f03c800d830 tx=0x7f03c800db40 comp rx=0 tx=0).stop 2023-07-06T21:42:43.665 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 shutdown_connections 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f03c006ace0 0x7f03c006d190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03d40c97f0 0x7f03d40b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03d40703a0 0x7f03d414cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 --2- 172.21.15.84:0/165310616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f03d40c8d50 0x7f03d40c6910 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 >> 172.21.15.84:0/165310616 conn(0x7f03d40cbe10 msgr2=0x7f03d40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 shutdown_connections 2023-07-06T21:42:43.666 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:43.661+0000 7f03d91ff700 1 -- 172.21.15.84:0/165310616 wait complete. 2023-07-06T21:42:43.667 INFO:teuthology.orchestra.run.smithi084.stderr:updated caps for client.0 2023-07-06T21:42:43.682 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-07-06T21:42:43.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:42:43.784 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-07-06T21:42:43.892 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-07-06T21:42:43.893 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:43.893 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:43.936 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:44.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.168+0000 7f0d74fff700 1 Processor -- start 2023-07-06T21:42:44.171 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.168+0000 7f0d74fff700 1 -- start start 2023-07-06T21:42:44.172 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.169+0000 7f0d74fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d700b83c0 0x7f0d700cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.169+0000 7f0d74fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0d700cfc60 0x7f0d700d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.169+0000 7f0d74fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d700d0a80 con 0x7f0d700d3710 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.169+0000 7f0d74fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d700b8790 con 0x7f0d700b83c0 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.169+0000 7f0d74fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d700b4e70 con 0x7f0d700cfc60 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d700b83c0 0x7f0d700cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0d700b83c0 0x7f0d700cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32768/0 (socket says 172.21.15.84:32768) 2023-07-06T21:42:44.173 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1949242108 learned_addr learned my addr 172.21.15.84:0/1949242108 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:44.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1949242108 >> v2:172.21.15.107:3300/0 conn(0x7f0d700cfc60 msgr2=0x7f0d700d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:44.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 --2- 172.21.15.84:0/1949242108 >> v2:172.21.15.107:3300/0 conn(0x7f0d700cfc60 0x7f0d700d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.175 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1949242108 >> v1:172.21.15.107:6789/0 conn(0x7f0d700d3710 legacy=0x7f0d700cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1949242108 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d700b5150 con 0x7f0d700b83c0 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.170+0000 7f0d6e59c700 1 --2- 172.21.15.84:0/1949242108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d700cd200 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0d700d0030 tx=0x7f0d60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fd69c8e665de2b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d6d59a700 1 -- 172.21.15.84:0/1949242108 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60004230 con 0x7f0d700b83c0 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d6d59a700 1 -- 172.21.15.84:0/1949242108 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d60004390 con 0x7f0d700b83c0 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d6d59a700 1 -- 172.21.15.84:0/1949242108 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600036a0 con 0x7f0d700b83c0 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 -- 172.21.15.84:0/1949242108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 msgr2=0x7f0d700cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.176 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1949242108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d700cd200 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0d700d0030 tx=0x7f0d60009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 -- 172.21.15.84:0/1949242108 shutdown_connections 2023-07-06T21:42:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1949242108 >> v2:172.21.15.107:3300/0 conn(0x7f0d700cfc60 0x7f0d700d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1949242108 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d700cd200 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 -- 172.21.15.84:0/1949242108 >> 172.21.15.84:0/1949242108 conn(0x7f0d7006b350 msgr2=0x7f0d7006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 -- 172.21.15.84:0/1949242108 shutdown_connections 2023-07-06T21:42:44.177 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.171+0000 7f0d74fff700 1 -- 172.21.15.84:0/1949242108 wait complete. 2023-07-06T21:42:44.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.173+0000 7f0d74fff700 1 Processor -- start 2023-07-06T21:42:44.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.173+0000 7f0d74fff700 1 -- start start 2023-07-06T21:42:44.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d74fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d70151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d74fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d700cfc60 0x7f0d70154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.179 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d74fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d700d3710 0x7f0d70156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d74fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70151690 con 0x7f0d700b83c0 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d74fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70153490 con 0x7f0d700cfc60 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d74fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70152880 con 0x7f0d700d3710 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.174+0000 7f0d6e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d70151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d70151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32776/0 (socket says 172.21.15.84:32776) 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1685022604 learned_addr learned my addr 172.21.15.84:0/1685022604 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:44.180 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d700d3710 msgr2=0x7f0d70156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d700d3710 0x7f0d70156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d700cfc60 msgr2=0x7f0d70154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6dd9b700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d700cfc60 0x7f0d70154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d700cfc60 0x7f0d70154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 -- 172.21.15.84:0/1685022604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009550 con 0x7f0d700b83c0 2023-07-06T21:42:44.181 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6dd9b700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d700cfc60 0x7f0d70154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d6e59c700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d70151150 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009130 tx=0x7f0d60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60011830 con 0x7f0d700b83c0 2023-07-06T21:42:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d60011990 con 0x7f0d700b83c0 2023-07-06T21:42:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d70152b60 con 0x7f0d700b83c0 2023-07-06T21:42:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.175+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60011d20 con 0x7f0d700b83c0 2023-07-06T21:42:44.182 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.176+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d70153120 con 0x7f0d700b83c0 2023-07-06T21:42:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.177+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d700b2430 con 0x7f0d700b83c0 2023-07-06T21:42:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.178+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0d6001b440 con 0x7f0d700b83c0 2023-07-06T21:42:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.178+0000 7f0d5effd700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0d5006ace0 0x7f0d5006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.178+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0d6008ac50 con 0x7f0d700b83c0 2023-07-06T21:42:44.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.179+0000 7f0d6dd9b700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0d5006ace0 0x7f0d5006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.179+0000 7f0d6dd9b700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0d5006ace0 0x7f0d5006d190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0d700cd320 tx=0x7f0d64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:44.186 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.183+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0d60055cf0 con 0x7f0d700b83c0 2023-07-06T21:42:44.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.380+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d700b2150 con 0x7f0d700b83c0 2023-07-06T21:42:44.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.381+0000 7f0d5effd700 1 -- 172.21.15.84:0/1685022604 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7f0d60059310 con 0x7f0d700b83c0 2023-07-06T21:42:44.385 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:44.385 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:44.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.383+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0d5006ace0 msgr2=0x7f0d5006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.383+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0d5006ace0 0x7f0d5006d190 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0d700cd320 tx=0x7f0d64009040 comp rx=0 tx=0).stop 2023-07-06T21:42:44.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.383+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 msgr2=0x7f0d70151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.383+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d70151150 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009130 tx=0x7f0d60004750 comp rx=0 tx=0).stop 2023-07-06T21:42:44.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 shutdown_connections 2023-07-06T21:42:44.387 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0d5006ace0 0x7f0d5006d190 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d700d3710 0x7f0d70156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0d700b83c0 0x7f0d70151150 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 --2- 172.21.15.84:0/1685022604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d700cfc60 0x7f0d70154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 >> 172.21.15.84:0/1685022604 conn(0x7f0d7006b350 msgr2=0x7f0d700b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:44.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 shutdown_connections 2023-07-06T21:42:44.388 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.384+0000 7f0d74fff700 1 -- 172.21.15.84:0/1685022604 wait complete. 2023-07-06T21:42:44.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:42:08.774391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:42:09.558402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:44.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.598+0000 7fc9363ff700 1 Processor -- start 2023-07-06T21:42:44.601 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.598+0000 7fc9363ff700 1 -- start start 2023-07-06T21:42:44.602 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.599+0000 7fc9363ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fc930070ab0 0x7fc93006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.599+0000 7fc9363ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300cf600 con 0x7fc9300ca8c0 2023-07-06T21:42:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.599+0000 7fc9363ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300ccf40 con 0x7fc930159220 2023-07-06T21:42:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.599+0000 7fc9363ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300cad60 con 0x7fc930070ab0 2023-07-06T21:42:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc92f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fc9300ca8c0 0x7fc9300703a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40632/0 (socket says 172.21.15.84:40632) 2023-07-06T21:42:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc92f7fe700 1 -- 172.21.15.84:0/1183827816 learned_addr learned my addr 172.21.15.84:0/1183827816 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:44.603 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc93499c700 1 --2- 172.21.15.84:0/1183827816 >> v2:172.21.15.84:3300/0 conn(0x7fc930070ab0 0x7fc93006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc92effd700 1 -- 172.21.15.84:0/1183827816 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590120575 0 0) 0x7fc9300cf600 con 0x7fc9300ca8c0 2023-07-06T21:42:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc92effd700 1 -- 172.21.15.84:0/1183827816 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9180057c0 con 0x7fc9300ca8c0 2023-07-06T21:42:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc93499c700 1 -- 172.21.15.84:0/1183827816 >> v1:172.21.15.107:6789/0 conn(0x7fc930159220 legacy=0x7fc9300cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc93499c700 1 -- 172.21.15.84:0/1183827816 >> v1:172.21.15.84:6789/0 conn(0x7fc9300ca8c0 legacy=0x7fc9300703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.605 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc93499c700 1 -- 172.21.15.84:0/1183827816 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300c9b00 con 0x7fc930070ab0 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.600+0000 7fc93499c700 1 --2- 172.21.15.84:0/1183827816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930070ab0 0x7fc93006c400 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367ae48422765556 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc92effd700 1 -- 172.21.15.84:0/1183827816 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800e070 con 0x7fc930070ab0 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc92effd700 1 -- 172.21.15.84:0/1183827816 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800bb20 con 0x7fc930070ab0 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc92effd700 1 -- 172.21.15.84:0/1183827816 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928012430 con 0x7fc930070ab0 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 -- 172.21.15.84:0/1183827816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930070ab0 msgr2=0x7fc93006c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 --2- 172.21.15.84:0/1183827816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930070ab0 0x7fc93006c400 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 -- 172.21.15.84:0/1183827816 shutdown_connections 2023-07-06T21:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 --2- 172.21.15.84:0/1183827816 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930070ab0 0x7fc93006c400 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 -- 172.21.15.84:0/1183827816 >> 172.21.15.84:0/1183827816 conn(0x7fc9300cbf10 msgr2=0x7fc9300ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:44.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 -- 172.21.15.84:0/1183827816 shutdown_connections 2023-07-06T21:42:44.607 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.601+0000 7fc9363ff700 1 -- 172.21.15.84:0/1183827816 wait complete. 2023-07-06T21:42:44.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.603+0000 7fc9363ff700 1 Processor -- start 2023-07-06T21:42:44.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.603+0000 7fc9363ff700 1 -- start start 2023-07-06T21:42:44.609 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc9363ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930070ab0 0x7fc9300ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc9363ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9300ca8c0 0x7fc93015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc9363ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 0x7fc9301ec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc9363ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93015b810 con 0x7fc930159220 2023-07-06T21:42:44.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc9363ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300b0130 con 0x7fc930070ab0 2023-07-06T21:42:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc9363ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300abfb0 con 0x7fc9300ca8c0 2023-07-06T21:42:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc93499c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 0x7fc9301ec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc93499c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 0x7fc9301ec0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32786/0 (socket says 172.21.15.84:32786) 2023-07-06T21:42:44.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.604+0000 7fc93499c700 1 -- 172.21.15.84:0/78312943 learned_addr learned my addr 172.21.15.84:0/78312943 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc92ffff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930070ab0 0x7fc9300ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc93499c700 1 -- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9300ca8c0 msgr2=0x7fc93015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.612 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc93499c700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9300ca8c0 0x7fc93015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc93499c700 1 -- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930070ab0 msgr2=0x7fc9300ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc93499c700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930070ab0 0x7fc9300ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.613 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc93499c700 1 -- 172.21.15.84:0/78312943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928009a70 con 0x7fc930159220 2023-07-06T21:42:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc92ffff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930070ab0 0x7fc9300ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc93499c700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 0x7fc9301ec0c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc92800b680 tx=0x7fc928004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928012430 con 0x7fc930159220 2023-07-06T21:42:44.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928020070 con 0x7fc930159220 2023-07-06T21:42:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9300ac290 con 0x7fc930159220 2023-07-06T21:42:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928004280 con 0x7fc930159220 2023-07-06T21:42:44.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.605+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9300ac850 con 0x7fc930159220 2023-07-06T21:42:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.606+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93004e1d0 con 0x7fc930159220 2023-07-06T21:42:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.607+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fc9280043e0 con 0x7fc930159220 2023-07-06T21:42:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.608+0000 7fc92cff9700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc91c06ace0 0x7fc91c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:44.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.608+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc928022070 con 0x7fc930159220 2023-07-06T21:42:44.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.608+0000 7fc92ffff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc91c06ace0 0x7fc91c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:44.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.609+0000 7fc92ffff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc91c06ace0 0x7fc91c06d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc920000f60 tx=0x7fc920009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:44.618 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.612+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fc9280bd310 con 0x7fc930159220 2023-07-06T21:42:44.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.808+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc930158880 con 0x7fc930159220 2023-07-06T21:42:44.811 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.808+0000 7fc92cff9700 1 -- 172.21.15.84:0/78312943 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc92802c020 con 0x7fc930159220 2023-07-06T21:42:44.812 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:44.813 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc91c06ace0 msgr2=0x7fc91c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc91c06ace0 0x7fc91c06d190 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc920000f60 tx=0x7fc920009040 comp rx=0 tx=0).stop 2023-07-06T21:42:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 msgr2=0x7fc9301ec0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 0x7fc9301ec0c0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc92800b680 tx=0x7fc928004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.814 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 shutdown_connections 2023-07-06T21:42:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fc91c06ace0 0x7fc91c06d190 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc930159220 0x7fc9301ec0c0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.815 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc930070ab0 0x7fc9300ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 --2- 172.21.15.84:0/78312943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9300ca8c0 0x7fc93015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.810+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 >> 172.21.15.84:0/78312943 conn(0x7fc9300cbf10 msgr2=0x7fc9300b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.811+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 shutdown_connections 2023-07-06T21:42:44.816 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:44.811+0000 7fc9363ff700 1 -- 172.21.15.84:0/78312943 wait complete. 2023-07-06T21:42:44.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:45.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.030+0000 7f5d165ff700 1 Processor -- start 2023-07-06T21:42:45.033 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.030+0000 7f5d165ff700 1 -- start start 2023-07-06T21:42:45.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.031+0000 7f5d165ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b81c0 0x7f5d100cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.031+0000 7f5d165ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d100ceda0 0x7f5d100cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.031+0000 7f5d165ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d100d1830 con 0x7f5d100b1a80 2023-07-06T21:42:45.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.031+0000 7f5d165ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d100cf330 con 0x7f5d100ceda0 2023-07-06T21:42:45.035 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.031+0000 7f5d165ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100d1270 con 0x7f5d100b81c0 2023-07-06T21:42:45.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d100ceda0 0x7f5d100cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5d100ceda0 0x7f5d100cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32788/0 (socket says 172.21.15.84:32788) 2023-07-06T21:42:45.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 -- 172.21.15.84:0/2525771840 learned_addr learned my addr 172.21.15.84:0/2525771840 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:45.036 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d0ffff700 1 --2- 172.21.15.84:0/2525771840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b81c0 0x7f5d100cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.037 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 -- 172.21.15.84:0/2525771840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b81c0 msgr2=0x7f5d100cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:45.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 --2- 172.21.15.84:0/2525771840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b81c0 0x7f5d100cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 -- 172.21.15.84:0/2525771840 >> v1:172.21.15.107:6789/0 conn(0x7f5d100b1a80 legacy=0x7f5d1015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d14b9c700 1 -- 172.21.15.84:0/2525771840 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100d1550 con 0x7f5d100ceda0 2023-07-06T21:42:45.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.032+0000 7f5d0f7fe700 1 -- 172.21.15.84:0/2525771840 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211797347 0 0) 0x7f5d100d1830 con 0x7f5d100b1a80 2023-07-06T21:42:45.038 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d14b9c700 1 --2- 172.21.15.84:0/2525771840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100ceda0 0x7f5d100cfb40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009e50 tx=0x7f5d0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75e8a87023c8a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d0f7fe700 1 -- 172.21.15.84:0/2525771840 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800e070 con 0x7f5d100ceda0 2023-07-06T21:42:45.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d0f7fe700 1 -- 172.21.15.84:0/2525771840 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800b910 con 0x7f5d100ceda0 2023-07-06T21:42:45.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d0f7fe700 1 -- 172.21.15.84:0/2525771840 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800bcc0 con 0x7f5d100ceda0 2023-07-06T21:42:45.039 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d165ff700 1 -- 172.21.15.84:0/2525771840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100ceda0 msgr2=0x7f5d100cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d165ff700 1 --2- 172.21.15.84:0/2525771840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100ceda0 0x7f5d100cfb40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009e50 tx=0x7f5d0800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:45.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d165ff700 1 -- 172.21.15.84:0/2525771840 shutdown_connections 2023-07-06T21:42:45.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d165ff700 1 --2- 172.21.15.84:0/2525771840 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100ceda0 0x7f5d100cfb40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.040 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d165ff700 1 --2- 172.21.15.84:0/2525771840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b81c0 0x7f5d100cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.033+0000 7f5d165ff700 1 -- 172.21.15.84:0/2525771840 >> 172.21.15.84:0/2525771840 conn(0x7f5d1006b2b0 msgr2=0x7f5d1006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:45.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.034+0000 7f5d165ff700 1 -- 172.21.15.84:0/2525771840 shutdown_connections 2023-07-06T21:42:45.041 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.034+0000 7f5d165ff700 1 -- 172.21.15.84:0/2525771840 wait complete. 2023-07-06T21:42:45.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.035+0000 7f5d165ff700 1 Processor -- start 2023-07-06T21:42:45.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.035+0000 7f5d165ff700 1 -- start start 2023-07-06T21:42:45.045 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.036+0000 7f5d165ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b1a80 0x7f5d100c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.036+0000 7f5d165ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 0x7f5d100c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.036+0000 7f5d165ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d100ceda0 0x7f5d101f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.036+0000 7f5d165ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100c8be0 con 0x7f5d100b81c0 2023-07-06T21:42:45.046 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.036+0000 7f5d165ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100c9670 con 0x7f5d100ceda0 2023-07-06T21:42:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.036+0000 7f5d165ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100c69c0 con 0x7f5d100b1a80 2023-07-06T21:42:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 0x7f5d100c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 0x7f5d100c86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32796/0 (socket says 172.21.15.84:32796) 2023-07-06T21:42:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 -- 172.21.15.84:0/1476450730 learned_addr learned my addr 172.21.15.84:0/1476450730 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d0ffff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b1a80 0x7f5d100c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d14b9c700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d100ceda0 0x7f5d101f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 -- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b1a80 msgr2=0x7f5d100c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b1a80 0x7f5d100c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 -- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d100ceda0 msgr2=0x7f5d101f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d100ceda0 0x7f5d101f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 -- 172.21.15.84:0/1476450730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08009b30 con 0x7f5d100b81c0 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d07fff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 0x7f5d100c86a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80011e0 tx=0x7f5cf800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.037+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf80054e0 con 0x7f5d100b81c0 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.038+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8003680 con 0x7f5d100b81c0 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.038+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8003a30 con 0x7f5d100b81c0 2023-07-06T21:42:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.038+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d100c8dd0 con 0x7f5d100b81c0 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.038+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d100c9390 con 0x7f5d100b81c0 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.039+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d1004e1d0 con 0x7f5d100b81c0 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.040+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5cf8003b90 con 0x7f5d100b81c0 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.041+0000 7f5d0d7fa700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5cf006ace0 0x7f5cf006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.041+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5cf8088970 con 0x7f5d100b81c0 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.041+0000 7f5d0ffff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5cf006ace0 0x7f5cf006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.042+0000 7f5d0ffff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5cf006ace0 0x7f5cf006d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009be0 tx=0x7f5d00009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.051 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.045+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5cf8053980 con 0x7f5d100b81c0 2023-07-06T21:42:45.245 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.242+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d100b4f30 con 0x7f5d100b81c0 2023-07-06T21:42:45.246 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.243+0000 7f5d0d7fa700 1 -- 172.21.15.84:0/1476450730 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5cf80170a0 con 0x7f5d100b81c0 2023-07-06T21:42:45.246 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:45.248 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5cf006ace0 msgr2=0x7f5cf006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5cf006ace0 0x7f5cf006d190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009be0 tx=0x7f5d00009210 comp rx=0 tx=0).stop 2023-07-06T21:42:45.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 msgr2=0x7f5d100c86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 0x7f5d100c86a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80011e0 tx=0x7f5cf800ca30 comp rx=0 tx=0).stop 2023-07-06T21:42:45.249 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 shutdown_connections 2023-07-06T21:42:45.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5cf006ace0 0x7f5cf006d190 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d100ceda0 0x7f5d101f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.250 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d100b1a80 0x7f5d100c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 --2- 172.21.15.84:0/1476450730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d100b81c0 0x7f5d100c86a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 >> 172.21.15.84:0/1476450730 conn(0x7f5d1006b2b0 msgr2=0x7f5d100b7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:45.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 shutdown_connections 2023-07-06T21:42:45.251 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.245+0000 7f5d165ff700 1 -- 172.21.15.84:0/1476450730 wait complete. 2023-07-06T21:42:45.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:42:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.459+0000 7fd3e1fff700 1 Processor -- start 2023-07-06T21:42:45.462 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.459+0000 7fd3e1fff700 1 -- start start 2023-07-06T21:42:45.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3e1fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3dc0b81c0 0x7fd3dc0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3e1fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd3dc0b1a80 0x7fd3dc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3e1fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3dc0d1830 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3e1fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc0cf330 con 0x7fd3dc0b81c0 2023-07-06T21:42:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3e1fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3dc0d1270 con 0x7fd3dc0b1a80 2023-07-06T21:42:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3db7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3dc0b81c0 0x7fd3dc0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3db7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd3dc0b81c0 0x7fd3dc0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32798/0 (socket says 172.21.15.84:32798) 2023-07-06T21:42:45.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3db7fe700 1 -- 172.21.15.84:0/4055738810 learned_addr learned my addr 172.21.15.84:0/4055738810 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.460+0000 7fd3daffd700 1 --2- 172.21.15.84:0/4055738810 >> v2:172.21.15.107:3300/0 conn(0x7fd3dc0b1a80 0x7fd3dc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3db7fe700 1 -- 172.21.15.84:0/4055738810 >> v2:172.21.15.107:3300/0 conn(0x7fd3dc0b1a80 msgr2=0x7fd3dc0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3db7fe700 1 --2- 172.21.15.84:0/4055738810 >> v2:172.21.15.107:3300/0 conn(0x7fd3dc0b1a80 0x7fd3dc0cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3db7fe700 1 -- 172.21.15.84:0/4055738810 >> v1:172.21.15.84:6789/0 conn(0x7fd3dc0ceda0 legacy=0x7fd3dc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3db7fe700 1 -- 172.21.15.84:0/4055738810 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3dc0d1550 con 0x7fd3dc0b81c0 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3db7fe700 1 --2- 172.21.15.84:0/4055738810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc0cf600 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0d0770 tx=0x7fd3cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0d1e548853731b server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3da7fc700 1 -- 172.21.15.84:0/4055738810 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc00b870 con 0x7fd3dc0b81c0 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3da7fc700 1 -- 172.21.15.84:0/4055738810 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc00b9d0 con 0x7fd3dc0b81c0 2023-07-06T21:42:45.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.461+0000 7fd3da7fc700 1 -- 172.21.15.84:0/4055738810 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc00bd60 con 0x7fd3dc0b81c0 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 -- 172.21.15.84:0/4055738810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0b81c0 msgr2=0x7fd3dc0cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/4055738810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc0cf600 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc0d0770 tx=0x7fd3cc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 -- 172.21.15.84:0/4055738810 shutdown_connections 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/4055738810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc0cf600 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/4055738810 >> v2:172.21.15.107:3300/0 conn(0x7fd3dc0b1a80 0x7fd3dc0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 -- 172.21.15.84:0/4055738810 >> 172.21.15.84:0/4055738810 conn(0x7fd3dc06b2b0 msgr2=0x7fd3dc06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 -- 172.21.15.84:0/4055738810 shutdown_connections 2023-07-06T21:42:45.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.462+0000 7fd3e1fff700 1 -- 172.21.15.84:0/4055738810 wait complete. 2023-07-06T21:42:45.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.464+0000 7fd3e1fff700 1 Processor -- start 2023-07-06T21:42:45.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.464+0000 7fd3e1fff700 1 -- start start 2023-07-06T21:42:45.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3dc0b1a80 0x7fd3dc155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc1f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3e1fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 0x7fd3dc1f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc1587b0 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc155cd0 con 0x7fd3dc0b81c0 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3e1fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3dc0cb660 con 0x7fd3dc0b1a80 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 0x7fd3dc1f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 0x7fd3dc1f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32804/0 (socket says 172.21.15.84:32804) 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 -- 172.21.15.84:0/2152067856 learned_addr learned my addr 172.21.15.84:0/2152067856 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3db7fe700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3dc0b1a80 0x7fd3dc155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3daffd700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc1f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 -- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3dc0b1a80 msgr2=0x7fd3dc155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3dc0b1a80 0x7fd3dc155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 -- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3dc0b81c0 msgr2=0x7fd3dc1f1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc1f1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3dbfff700 1 -- 172.21.15.84:0/2152067856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc009550 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.465+0000 7fd3db7fe700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3dc0b1a80 0x7fd3dc155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.466+0000 7fd3dbfff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 0x7fd3dc1f2210 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400cc50 tx=0x7fd3d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.466+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d400ad80 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.466+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d400ce50 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.466+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3dc0cb940 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.466+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d4003c40 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.466+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3dc0cbf00 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.467+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3dc04e1d0 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.468+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd3d40050d0 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.469+0000 7fd3c3fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd3c806ace0 0x7fd3c806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.469+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd3d4089630 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.469+0000 7fd3db7fe700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd3c806ace0 0x7fd3c806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.470+0000 7fd3db7fe700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd3c806ace0 0x7fd3c806d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc000c00 tx=0x7fd3cc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.473+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fd3d4054640 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.670 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.667+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3dc0b4f30 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.668+0000 7fd3c3fff700 1 -- 172.21.15.84:0/2152067856 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3d4057c60 con 0x7fd3dc0ceda0 2023-07-06T21:42:45.671 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:42:45.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd3c806ace0 msgr2=0x7fd3c806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd3c806ace0 0x7fd3c806d190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc000c00 tx=0x7fd3cc008030 comp rx=0 tx=0).stop 2023-07-06T21:42:45.673 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 msgr2=0x7fd3dc1f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 0x7fd3dc1f2210 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd3d400cc50 tx=0x7fd3d400a340 comp rx=0 tx=0).stop 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 shutdown_connections 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd3c806ace0 0x7fd3c806d190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3dc0ceda0 0x7fd3dc1f2210 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3dc0b1a80 0x7fd3dc155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 --2- 172.21.15.84:0/2152067856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3dc0b81c0 0x7fd3dc1f1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 >> 172.21.15.84:0/2152067856 conn(0x7fd3dc06b2b0 msgr2=0x7fd3dc0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:45.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 shutdown_connections 2023-07-06T21:42:45.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.670+0000 7fd3e1fff700 1 -- 172.21.15.84:0/2152067856 wait complete. 2023-07-06T21:42:45.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:42:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.884+0000 7f9ed8dff700 1 Processor -- start 2023-07-06T21:42:45.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.884+0000 7f9ed8dff700 1 -- start start 2023-07-06T21:42:45.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.885+0000 7f9ed8dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40b81c0 0x7f9ed40cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.885+0000 7f9ed8dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40d1830 con 0x7f9ed40ceda0 2023-07-06T21:42:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed8dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed4160d40 con 0x7f9ed40b1a80 2023-07-06T21:42:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed8dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40cf330 con 0x7f9ed40b81c0 2023-07-06T21:42:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed2d9d700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f9ed40ceda0 0x7f9ed4160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40646/0 (socket says 172.21.15.84:40646) 2023-07-06T21:42:45.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed2d9d700 1 -- 172.21.15.84:0/3089604216 learned_addr learned my addr 172.21.15.84:0/3089604216 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed259c700 1 --2- 172.21.15.84:0/3089604216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40b81c0 0x7f9ed40cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864257352 0 0) 0x7f9ed40d1830 con 0x7f9ed40ceda0 2023-07-06T21:42:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ebc0035b0 con 0x7f9ed40ceda0 2023-07-06T21:42:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 283782316 0 0) 0x7f9ed4160d40 con 0x7f9ed40b1a80 2023-07-06T21:42:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed40d1830 con 0x7f9ed40b1a80 2023-07-06T21:42:45.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed259c700 1 -- 172.21.15.84:0/3089604216 >> v1:172.21.15.107:6789/0 conn(0x7f9ed40b1a80 legacy=0x7f9ed415ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed259c700 1 -- 172.21.15.84:0/3089604216 >> v1:172.21.15.84:6789/0 conn(0x7f9ed40ceda0 legacy=0x7f9ed4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.886+0000 7f9ed259c700 1 -- 172.21.15.84:0/3089604216 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40d1270 con 0x7f9ed40b81c0 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.887+0000 7f9ed259c700 1 --2- 172.21.15.84:0/3089604216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40b81c0 0x7f9ed40cf600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009870 tx=0x7f9ec4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc1f819d0096485 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.887+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4011830 con 0x7f9ed40b81c0 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.887+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4011990 con 0x7f9ed40b81c0 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.887+0000 7f9ed159a700 1 -- 172.21.15.84:0/3089604216 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4011d00 con 0x7f9ed40b81c0 2023-07-06T21:42:45.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 -- 172.21.15.84:0/3089604216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40b81c0 msgr2=0x7f9ed40cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/3089604216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40b81c0 0x7f9ed40cf600 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4009870 tx=0x7f9ec4009b80 comp rx=0 tx=0).stop 2023-07-06T21:42:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 -- 172.21.15.84:0/3089604216 shutdown_connections 2023-07-06T21:42:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/3089604216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40b81c0 0x7f9ed40cf600 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 -- 172.21.15.84:0/3089604216 >> 172.21.15.84:0/3089604216 conn(0x7f9ed406b2b0 msgr2=0x7f9ed406b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 -- 172.21.15.84:0/3089604216 shutdown_connections 2023-07-06T21:42:45.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.888+0000 7f9ed8dff700 1 -- 172.21.15.84:0/3089604216 wait complete. 2023-07-06T21:42:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.889+0000 7f9ed8dff700 1 Processor -- start 2023-07-06T21:42:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.890+0000 7f9ed8dff700 1 -- start start 2023-07-06T21:42:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.890+0000 7f9ed8dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed40b1a80 0x7f9ed40cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.890+0000 7f9ed8dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 0x7f9ed40cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.890+0000 7f9ed8dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40ceda0 0x7f9ed40c75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed8dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40ca160 con 0x7f9ed40b81c0 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed8dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40ca440 con 0x7f9ed40b1a80 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed8dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40ca720 con 0x7f9ed40ceda0 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 0x7f9ed40cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 0x7f9ed40cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32816/0 (socket says 172.21.15.84:32816) 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 -- 172.21.15.84:0/1614334763 learned_addr learned my addr 172.21.15.84:0/1614334763 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:45.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed259c700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed40b1a80 0x7f9ed40cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed2d9d700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40ceda0 0x7f9ed40c75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 -- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40ceda0 msgr2=0x7f9ed40c75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40ceda0 0x7f9ed40c75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 -- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed40b1a80 msgr2=0x7f9ed40cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed40b1a80 0x7f9ed40cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:45.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 -- 172.21.15.84:0/1614334763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec40094a0 con 0x7f9ed40b81c0 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed259c700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed40b1a80 0x7f9ed40cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.891+0000 7f9ed1d9b700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 0x7f9ed40cc0e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c500 tx=0x7f9ec800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.892+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80052d0 con 0x7f9ed40b81c0 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.892+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec8005430 con 0x7f9ed40b81c0 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.892+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40caa00 con 0x7f9ed40b81c0 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.892+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80038c0 con 0x7f9ed40b81c0 2023-07-06T21:42:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.892+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed4154650 con 0x7f9ed40b81c0 2023-07-06T21:42:45.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.893+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed40caf20 con 0x7f9ed40b81c0 2023-07-06T21:42:45.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.894+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9ec8003a20 con 0x7f9ed40b81c0 2023-07-06T21:42:45.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.895+0000 7f9ec2ffd700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9eb406ace0 0x7f9eb406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:45.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.895+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9ec8088ab0 con 0x7f9ed40b81c0 2023-07-06T21:42:45.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.895+0000 7f9ed259c700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9eb406ace0 0x7f9eb406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:45.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.896+0000 7f9ed259c700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9eb406ace0 0x7f9eb406d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40090b0 tx=0x7f9ec4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:45.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:45.899+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9ed40caf20 con 0x7f9ed40b81c0 2023-07-06T21:42:46.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.095+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ed40caf20 con 0x7f9ed40b81c0 2023-07-06T21:42:46.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.095+0000 7f9ec2ffd700 1 -- 172.21.15.84:0/1614334763 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ec8017020 con 0x7f9ed40b81c0 2023-07-06T21:42:46.098 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:42:46.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9eb406ace0 msgr2=0x7f9eb406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9eb406ace0 0x7f9eb406d190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40090b0 tx=0x7f9ec4008030 comp rx=0 tx=0).stop 2023-07-06T21:42:46.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 msgr2=0x7f9ed40cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 0x7f9ed40cc0e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800c500 tx=0x7f9ec800c8c0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 shutdown_connections 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9eb406ace0 0x7f9eb406d190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40ceda0 0x7f9ed40c75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed40b1a80 0x7f9ed40cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 --2- 172.21.15.84:0/1614334763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ed40b81c0 0x7f9ed40cc0e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 >> 172.21.15.84:0/1614334763 conn(0x7f9ed406b2b0 msgr2=0x7f9ed415c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 shutdown_connections 2023-07-06T21:42:46.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.098+0000 7f9ed8dff700 1 -- 172.21.15.84:0/1614334763 wait complete. 2023-07-06T21:42:46.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-06T21:42:46.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.312+0000 7feb371ff700 1 Processor -- start 2023-07-06T21:42:46.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.313+0000 7feb371ff700 1 -- start start 2023-07-06T21:42:46.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb371ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb300b8530 0x7feb300cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:46.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb371ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:46.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb371ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb300d0990 con 0x7feb300cf3b0 2023-07-06T21:42:46.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb371ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300d3af0 con 0x7feb300b8530 2023-07-06T21:42:46.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb371ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300cd9d0 con 0x7feb300d3580 2023-07-06T21:42:46.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb300b8530 0x7feb300cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:46.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb300b8530 0x7feb300cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32820/0 (socket says 172.21.15.84:32820) 2023-07-06T21:42:46.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 -- 172.21.15.84:0/1193254804 learned_addr learned my addr 172.21.15.84:0/1193254804 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:46.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb2ffff700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:46.319 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 -- 172.21.15.84:0/1193254804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 msgr2=0x7feb300d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:46.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 -- 172.21.15.84:0/1193254804 >> v1:172.21.15.107:6789/0 conn(0x7feb300cf3b0 legacy=0x7feb3015feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.314+0000 7feb34f9b700 1 -- 172.21.15.84:0/1193254804 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300cd410 con 0x7feb300b8530 2023-07-06T21:42:46.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.315+0000 7feb2ffff700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:46.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.315+0000 7feb34f9b700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300cee70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7feb240098e0 tx=0x7feb24009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf074cc3349dcc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:46.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.315+0000 7feb2f7fe700 1 -- 172.21.15.84:0/1193254804 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2400b8a0 con 0x7feb300b8530 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.315+0000 7feb2f7fe700 1 -- 172.21.15.84:0/1193254804 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2400ba00 con 0x7feb300b8530 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.315+0000 7feb2f7fe700 1 -- 172.21.15.84:0/1193254804 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2400bdb0 con 0x7feb300b8530 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 -- 172.21.15.84:0/1193254804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 msgr2=0x7feb300cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300cee70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7feb240098e0 tx=0x7feb24009bf0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 -- 172.21.15.84:0/1193254804 shutdown_connections 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300cee70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 --2- 172.21.15.84:0/1193254804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 -- 172.21.15.84:0/1193254804 >> 172.21.15.84:0/1193254804 conn(0x7feb300715c0 msgr2=0x7feb30071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:46.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 -- 172.21.15.84:0/1193254804 shutdown_connections 2023-07-06T21:42:46.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.316+0000 7feb371ff700 1 -- 172.21.15.84:0/1193254804 wait complete. 2023-07-06T21:42:46.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.317+0000 7feb371ff700 1 Processor -- start 2023-07-06T21:42:46.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.318+0000 7feb371ff700 1 -- start start 2023-07-06T21:42:46.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb371ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:46.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb371ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb300cf3b0 0x7feb300ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:46.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb371ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:46.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb371ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300ca920 con 0x7feb300b8530 2023-07-06T21:42:46.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb371ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300b36d0 con 0x7feb300cf3b0 2023-07-06T21:42:46.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb371ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb300cccc0 con 0x7feb300d3580 2023-07-06T21:42:46.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32832/0 (socket says 172.21.15.84:32832) 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 -- 172.21.15.84:0/126093186 learned_addr learned my addr 172.21.15.84:0/126093186 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb2ffff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb300cf3b0 0x7feb300ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb3579c700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 -- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 msgr2=0x7feb300b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 -- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb300cf3b0 msgr2=0x7feb300ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.326 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb300cf3b0 0x7feb300ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 -- 172.21.15.84:0/126093186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240094f0 con 0x7feb300b8530 2023-07-06T21:42:46.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb2ffff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb300cf3b0 0x7feb300ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:46.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb3579c700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:46.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.319+0000 7feb34f9b700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300b4400 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7feb24013000 tx=0x7feb24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:46.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.320+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24011d50 con 0x7feb300b8530 2023-07-06T21:42:46.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.320+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2401f070 con 0x7feb300b8530 2023-07-06T21:42:46.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.320+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb300b3920 con 0x7feb300b8530 2023-07-06T21:42:46.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.320+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb240266c0 con 0x7feb300b8530 2023-07-06T21:42:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.320+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb300b3ee0 con 0x7feb300b8530 2023-07-06T21:42:46.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.321+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3004e240 con 0x7feb300b8530 2023-07-06T21:42:46.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.322+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb2401a430 con 0x7feb300b8530 2023-07-06T21:42:46.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.323+0000 7feb2d7fa700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb1406ad10 0x7feb1406d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:46.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.323+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feb24021070 con 0x7feb300b8530 2023-07-06T21:42:46.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.323+0000 7feb2ffff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb1406ad10 0x7feb1406d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:46.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.324+0000 7feb2ffff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb1406ad10 0x7feb1406d1c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb301f56c0 tx=0x7feb1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:46.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.327+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feb240561f0 con 0x7feb300b8530 2023-07-06T21:42:46.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.536+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb30072320 con 0x7feb300b8530 2023-07-06T21:42:46.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.536+0000 7feb2d7fa700 1 -- 172.21.15.84:0/126093186 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v53) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb2402b080 con 0x7feb300b8530 2023-07-06T21:42:46.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb1406ad10 msgr2=0x7feb1406d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb1406ad10 0x7feb1406d1c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7feb301f56c0 tx=0x7feb1c008040 comp rx=0 tx=0).stop 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 msgr2=0x7feb300b4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300b4400 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7feb24013000 tx=0x7feb24004750 comp rx=0 tx=0).stop 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 shutdown_connections 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb1406ad10 0x7feb1406d1c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb300d3580 0x7feb300b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb300b8530 0x7feb300b4400 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 --2- 172.21.15.84:0/126093186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb300cf3b0 0x7feb300ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:46.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 >> 172.21.15.84:0/126093186 conn(0x7feb300715c0 msgr2=0x7feb300732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:46.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 shutdown_connections 2023-07-06T21:42:46.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:46.539+0000 7feb371ff700 1 -- 172.21.15.84:0/126093186 wait complete. 2023-07-06T21:42:46.544 INFO:teuthology.orchestra.run.smithi084.stderr:listed 0 entries 2023-07-06T21:42:46.560 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi084.front.sepia.ceph.com... 2023-07-06T21:42:46.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:46.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-07-06T21:42:46.625 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi084.stderr:ceph-fuse[156108]: fuse finished with error 0 and tester_r 0 2023-07-06T21:42:46.628 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:42:46.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:46.629 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-07-06T21:42:46.785 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:42:46.786 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:46.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:42:46.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:42:46.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:46.834 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-07-06T21:42:46.918 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-07-06T21:42:46.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:42:46.919 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:46.919 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:46.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:42:46.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:42:46.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:46.964 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-07-06T21:42:47.044 INFO:teuthology.orchestra.run.smithi084.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-07-06T21:42:47.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:42:47.047 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:42:47.047 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:47.093 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:42:47.093 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:42:47.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:47.094 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-07-06T21:42:47.176 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-07-06T21:42:47.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:42:47.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.368+0000 7ff4ba3ff700 1 Processor -- start 2023-07-06T21:42:47.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.368+0000 7ff4ba3ff700 1 -- start start 2023-07-06T21:42:47.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4ba3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff4b40b59c0 0x7ff4b4166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4ba3ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40cff80 con 0x7ff4b40b1fd0 2023-07-06T21:42:47.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4ba3ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40d1460 con 0x7ff4b40d2690 2023-07-06T21:42:47.372 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4ba3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40d2530 con 0x7ff4b40b59c0 2023-07-06T21:42:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4b3fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff4b40b1fd0 0x7ff4b40cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40654/0 (socket says 172.21.15.84:40654) 2023-07-06T21:42:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4b3fff700 1 -- 172.21.15.84:0/4147127014 learned_addr learned my addr 172.21.15.84:0/4147127014 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:47.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4b899c700 1 --2- 172.21.15.84:0/4147127014 >> v2:172.21.15.107:3300/0 conn(0x7ff4b40b59c0 0x7ff4b4166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.369+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52109322 0 0) 0x7ff4a4001110 con 0x7ff4b40b1fd0 2023-07-06T21:42:47.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff49c0035b0 con 0x7ff4b40b1fd0 2023-07-06T21:42:47.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2662527302 0 0) 0x7ff49c0035b0 con 0x7ff4b40b1fd0 2023-07-06T21:42:47.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 >> v2:172.21.15.107:3300/0 conn(0x7ff4b40b59c0 msgr2=0x7ff4b4166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:47.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 --2- 172.21.15.84:0/4147127014 >> v2:172.21.15.107:3300/0 conn(0x7ff4b40b59c0 0x7ff4b4166fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 >> v1:172.21.15.107:6789/0 conn(0x7ff4b40d2690 legacy=0x7ff4b40cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40d1740 con 0x7ff4b40b1fd0 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7ff4a4001800 con 0x7ff4b40b1fd0 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b899c700 1 --2- 172.21.15.84:0/4147127014 >> v2:172.21.15.107:3300/0 conn(0x7ff4b40b59c0 0x7ff4b4166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 >> v1:172.21.15.84:6789/0 conn(0x7ff4b40b1fd0 legacy=0x7ff4b40cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c003a00 0x7ff49c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c004850 0x7ff49c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c005200 0x7ff49c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40d2530 con 0x7ff49c005200 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c00c110 con 0x7ff49c003a00 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c00c2d0 con 0x7ff49c004850 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b899c700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c005200 0x7ff49c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c003a00 0x7ff49c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.370+0000 7ff4b37fe700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c004850 0x7ff49c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b3fff700 1 -- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c004850 msgr2=0x7ff49c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c004850 0x7ff49c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b3fff700 1 -- 172.21.15.84:0/4147127014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c005200 msgr2=0x7ff49c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c005200 0x7ff49c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b3fff700 1 -- 172.21.15.84:0/4147127014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0035b0 con 0x7ff49c003a00 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b899c700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c005200 0x7ff49c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b899c700 1 -- 172.21.15.84:0/4147127014 reap_dead start 2023-07-06T21:42:47.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c003a00 0x7ff49c003e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c240 tx=0x7ff49c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5303908ad55b58 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4007db0 con 0x7ff49c003a00 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a400ce00 con 0x7ff49c003a00 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.371+0000 7ff4b2ffd700 1 -- 172.21.15.84:0/4147127014 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4005ba0 con 0x7ff49c003a00 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c003a00 msgr2=0x7ff49c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c003a00 0x7ff49c003e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c240 tx=0x7ff49c005880 comp rx=0 tx=0).stop 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/4147127014 shutdown_connections 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/4147127014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c003a00 0x7ff49c003e10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/4147127014 >> 172.21.15.84:0/4147127014 conn(0x7ff4b406b340 msgr2=0x7ff4b406b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:47.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/4147127014 shutdown_connections 2023-07-06T21:42:47.380 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.372+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/4147127014 wait complete. 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.374+0000 7ff4ba3ff700 1 Processor -- start 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.374+0000 7ff4ba3ff700 1 -- start start 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4ba3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 0x7ff4b414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4ba3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4b415ec80 0x7ff4b414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4ba3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b40b7e10 0x7ff4b41513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4ba3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414c930 con 0x7ff49c003a00 2023-07-06T21:42:47.382 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4ba3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c9ab0 con 0x7ff4b40b7e10 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4ba3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414d730 con 0x7ff4b415ec80 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 0x7ff4b414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 0x7ff4b414ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32846/0 (socket says 172.21.15.84:32846) 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 -- 172.21.15.84:0/3285504402 learned_addr learned my addr 172.21.15.84:0/3285504402 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b899c700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b40b7e10 0x7ff4b41513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 -- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4b415ec80 msgr2=0x7ff4b414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.383 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b37fe700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4b415ec80 0x7ff4b414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4b415ec80 0x7ff4b414c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 -- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b40b7e10 msgr2=0x7ff4b41513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b40b7e10 0x7ff4b41513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b3fff700 1 -- 172.21.15.84:0/3285504402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b415f0f0 con 0x7ff49c003a00 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b899c700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b40b7e10 0x7ff4b41513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.375+0000 7ff4b37fe700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4b415ec80 0x7ff4b414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.376+0000 7ff4b3fff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 0x7ff4b414ee60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c2a0 tx=0x7ff4a4001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:47.384 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.376+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4004c80 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.376+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4004de0 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.376+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b414cb20 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.376+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40072d0 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.376+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b414d0e0 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.377+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40b7680 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.378+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff4a4007430 con 0x7ff49c003a00 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.379+0000 7ff4b0ff9700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff4a006ace0 0x7ff4a006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.385 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.379+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff4a4089f50 con 0x7ff49c003a00 2023-07-06T21:42:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.379+0000 7ff4b37fe700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff4a006ace0 0x7ff4a006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.379+0000 7ff4b37fe700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff4a006ace0 0x7ff4a006d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff4b414b8b0 tx=0x7ff4a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:47.386 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.383+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff4a4054ff0 con 0x7ff49c003a00 2023-07-06T21:42:47.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.620+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4b40cff80 con 0x7ff49c003a00 2023-07-06T21:42:47.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.621+0000 7ff4b0ff9700 1 -- 172.21.15.84:0/3285504402 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4674 (secure 0 0 0) 0x7ff4b40cff80 con 0x7ff49c003a00 2023-07-06T21:42:47.624 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:47.625 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:42:10.578278+0000","modified":"2023-07-06T21:42:13.588255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6834/2341549811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2341549811},{"type":"v1","addr":"172.21.15.107:6834","nonce":2341549811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T21:42:47.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff4a006ace0 msgr2=0x7ff4a006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff4a006ace0 0x7ff4a006d190 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff4b414b8b0 tx=0x7ff4a8009040 comp rx=0 tx=0).stop 2023-07-06T21:42:47.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 msgr2=0x7ff4b414ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 0x7ff4b414ee60 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c2a0 tx=0x7ff4a4001b90 comp rx=0 tx=0).stop 2023-07-06T21:42:47.627 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 shutdown_connections 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff4a006ace0 0x7ff4a006d190 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff4b40b7e10 0x7ff4b41513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff49c003a00 0x7ff4b414ee60 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 --2- 172.21.15.84:0/3285504402 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4b415ec80 0x7ff4b414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 >> 172.21.15.84:0/3285504402 conn(0x7ff4b406b340 msgr2=0x7ff4b40cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 shutdown_connections 2023-07-06T21:42:47.628 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.624+0000 7ff4ba3ff700 1 -- 172.21.15.84:0/3285504402 wait complete. 2023-07-06T21:42:47.629 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-07-06T21:42:47.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:47.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.833+0000 7fad7c9ff700 1 Processor -- start 2023-07-06T21:42:47.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.833+0000 7fad7c9ff700 1 -- start start 2023-07-06T21:42:47.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7c9ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad780d1580 0x7fad78166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7c9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780d0990 con 0x7fad780b8530 2023-07-06T21:42:47.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7c9ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780cd9d0 con 0x7fad780d3580 2023-07-06T21:42:47.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7c9ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780d3af0 con 0x7fad780d1580 2023-07-06T21:42:47.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7659c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fad780b8530 0x7fad780cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40670/0 (socket says 172.21.15.84:40670) 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7659c700 1 -- 172.21.15.84:0/907865167 learned_addr learned my addr 172.21.15.84:0/907865167 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad76d9d700 1 --2- 172.21.15.84:0/907865167 >> v2:172.21.15.107:3300/0 conn(0x7fad780d1580 0x7fad78166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137029041 0 0) 0x7fad780d0990 con 0x7fad780b8530 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad600035b0 con 0x7fad780b8530 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2691528539 0 0) 0x7fad780cd9d0 con 0x7fad780d3580 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad780d0990 con 0x7fad780d3580 2023-07-06T21:42:47.844 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad76d9d700 1 -- 172.21.15.84:0/907865167 >> v1:172.21.15.107:6789/0 conn(0x7fad780d3580 legacy=0x7fad780d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad76d9d700 1 -- 172.21.15.84:0/907865167 >> v1:172.21.15.84:6789/0 conn(0x7fad780b8530 legacy=0x7fad780cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.835+0000 7fad76d9d700 1 -- 172.21.15.84:0/907865167 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780cd410 con 0x7fad780d1580 2023-07-06T21:42:47.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.836+0000 7fad76d9d700 1 --2- 172.21.15.84:0/907865167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d1580 0x7fad78166ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b7cf7e7e4fbbee9 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:47.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.836+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000e070 con 0x7fad780d1580 2023-07-06T21:42:47.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.836+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000bb20 con 0x7fad780d1580 2023-07-06T21:42:47.845 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.836+0000 7fad7559a700 1 -- 172.21.15.84:0/907865167 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70012430 con 0x7fad780d1580 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.836+0000 7fad7c9ff700 1 -- 172.21.15.84:0/907865167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d1580 msgr2=0x7fad78166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.837+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/907865167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d1580 0x7fad78166ff0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7000b330 comp rx=0 tx=0).stop 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.837+0000 7fad7c9ff700 1 -- 172.21.15.84:0/907865167 shutdown_connections 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.837+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/907865167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d1580 0x7fad78166ff0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.837+0000 7fad7c9ff700 1 -- 172.21.15.84:0/907865167 >> 172.21.15.84:0/907865167 conn(0x7fad780715c0 msgr2=0x7fad78071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.837+0000 7fad7c9ff700 1 -- 172.21.15.84:0/907865167 shutdown_connections 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.837+0000 7fad7c9ff700 1 -- 172.21.15.84:0/907865167 wait complete. 2023-07-06T21:42:47.846 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.838+0000 7fad7c9ff700 1 Processor -- start 2023-07-06T21:42:47.847 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 -- start start 2023-07-06T21:42:47.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad780b8530 0x7fad780cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 0x7fad780a9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d3580 0x7fad780c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780cb510 con 0x7fad780d1580 2023-07-06T21:42:47.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780a9a50 con 0x7fad780d3580 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.839+0000 7fad7c9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780ca900 con 0x7fad780b8530 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 0x7fad780a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 0x7fad780a9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32860/0 (socket says 172.21.15.84:32860) 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 -- 172.21.15.84:0/3746146380 learned_addr learned my addr 172.21.15.84:0/3746146380 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad7659c700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad780b8530 0x7fad780cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad76d9d700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d3580 0x7fad780c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 -- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad780b8530 msgr2=0x7fad780cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad780b8530 0x7fad780cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 -- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d3580 msgr2=0x7fad780c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d3580 0x7fad780c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 -- 172.21.15.84:0/3746146380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009a70 con 0x7fad780d1580 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.840+0000 7fad75d9b700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 0x7fad780a9510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c500 tx=0x7fad6c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.841+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0052d0 con 0x7fad780d1580 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.841+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c005450 con 0x7fad780d1580 2023-07-06T21:42:47.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.841+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad780cabe0 con 0x7fad780d1580 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.841+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0038c0 con 0x7fad780d1580 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.841+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad780cb1a0 con 0x7fad780d1580 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.842+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad780cea50 con 0x7fad780d1580 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.843+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fad6c003a20 con 0x7fad780d1580 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.844+0000 7fad66ffd700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fad5806ace0 0x7fad5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.844+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad6c0888b0 con 0x7fad780d1580 2023-07-06T21:42:47.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.844+0000 7fad7659c700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fad5806ace0 0x7fad5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:47.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.845+0000 7fad7659c700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fad5806ace0 0x7fad5806d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad680010e0 tx=0x7fad68007c70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:47.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:47.848+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fad6c0538c0 con 0x7fad780d1580 2023-07-06T21:42:48.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.046+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad780cf530 con 0x7fad780d1580 2023-07-06T21:42:48.050 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.048+0000 7fad66ffd700 1 -- 172.21.15.84:0/3746146380 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7fad6c056ee0 con 0x7fad780d1580 2023-07-06T21:42:48.051 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:48.051 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:48.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fad5806ace0 msgr2=0x7fad5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fad5806ace0 0x7fad5806d190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fad680010e0 tx=0x7fad68007c70 comp rx=0 tx=0).stop 2023-07-06T21:42:48.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 msgr2=0x7fad780a9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 0x7fad780a9510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c500 tx=0x7fad6c00c810 comp rx=0 tx=0).stop 2023-07-06T21:42:48.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 shutdown_connections 2023-07-06T21:42:48.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fad5806ace0 0x7fad5806d190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad780d3580 0x7fad780c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad780b8530 0x7fad780cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 --2- 172.21.15.84:0/3746146380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fad780d1580 0x7fad780a9510 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 >> 172.21.15.84:0/3746146380 conn(0x7fad780715c0 msgr2=0x7fad78073410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 shutdown_connections 2023-07-06T21:42:48.055 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.051+0000 7fad7c9ff700 1 -- 172.21.15.84:0/3746146380 wait complete. 2023-07-06T21:42:48.072 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-06T21:39:12.175836+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-07-06T21:39:18.713799+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-06T21:42:08.774391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-07-06T21:42:09.558402+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-06T21:42:48.073 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-06T21:42:48.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.268+0000 7fec2b5ff700 1 Processor -- start 2023-07-06T21:42:48.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.268+0000 7fec2b5ff700 1 -- start start 2023-07-06T21:42:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.268+0000 7fec2b5ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec2415c750 0x7fec240ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.268+0000 7fec2b5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240cf690 con 0x7fec240ad960 2023-07-06T21:42:48.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.268+0000 7fec2b5ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec240ac8c0 con 0x7fec240ac450 2023-07-06T21:42:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.268+0000 7fec2b5ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec240acba0 con 0x7fec2415c750 2023-07-06T21:42:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec2415c750 0x7fec240ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fec2415c750 0x7fec240ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32874/0 (socket says 172.21.15.84:32874) 2023-07-06T21:42:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 -- 172.21.15.84:0/2670931131 learned_addr learned my addr 172.21.15.84:0/2670931131 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec29b9c700 1 --1- 172.21.15.84:0/2670931131 >> v1:172.21.15.84:6789/0 conn(0x7fec240ad960 0x7fec240add60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40680/0 (socket says 172.21.15.84:40680) 2023-07-06T21:42:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 -- 172.21.15.84:0/2670931131 >> v1:172.21.15.107:6789/0 conn(0x7fec240ac450 legacy=0x7fec240c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 -- 172.21.15.84:0/2670931131 >> v1:172.21.15.84:6789/0 conn(0x7fec240ad960 legacy=0x7fec240add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 -- 172.21.15.84:0/2670931131 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240ace80 con 0x7fec2415c750 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec1bfff700 1 -- 172.21.15.84:0/2670931131 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940935123 0 0) 0x7fec240cf690 con 0x7fec240ad960 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.269+0000 7fec2939b700 1 --2- 172.21.15.84:0/2670931131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec240ce470 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fec20006fd0 tx=0x7fec20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a669a5b1aa5671b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec1bfff700 1 -- 172.21.15.84:0/2670931131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20005e30 con 0x7fec2415c750 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec1bfff700 1 -- 172.21.15.84:0/2670931131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20004030 con 0x7fec2415c750 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec1bfff700 1 -- 172.21.15.84:0/2670931131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec200043e0 con 0x7fec2415c750 2023-07-06T21:42:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 -- 172.21.15.84:0/2670931131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 msgr2=0x7fec240ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/2670931131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec240ce470 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fec20006fd0 tx=0x7fec20009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 -- 172.21.15.84:0/2670931131 shutdown_connections 2023-07-06T21:42:48.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/2670931131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec240ce470 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 -- 172.21.15.84:0/2670931131 >> 172.21.15.84:0/2670931131 conn(0x7fec2406b8c0 msgr2=0x7fec240cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 -- 172.21.15.84:0/2670931131 shutdown_connections 2023-07-06T21:42:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.270+0000 7fec2b5ff700 1 -- 172.21.15.84:0/2670931131 wait complete. 2023-07-06T21:42:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.272+0000 7fec2b5ff700 1 Processor -- start 2023-07-06T21:42:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.272+0000 7fec2b5ff700 1 -- start start 2023-07-06T21:42:48.277 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2b5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec240ac450 0x7fec2415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2b5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec240ad960 0x7fec24158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2b5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec24158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2b5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec240c7fd0 con 0x7fec2415c750 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2b5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec24158e90 con 0x7fec240ac450 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2b5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec240c7350 con 0x7fec240ad960 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec29b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec24158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.278 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec29b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec24158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32878/0 (socket says 172.21.15.84:32878) 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec29b9c700 1 -- 172.21.15.84:0/1048704230 learned_addr learned my addr 172.21.15.84:0/1048704230 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec2939b700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec240ac450 0x7fec2415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.273+0000 7fec28b9a700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec240ad960 0x7fec24158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec29b9c700 1 -- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec240ad960 msgr2=0x7fec24158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec29b9c700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec240ad960 0x7fec24158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec29b9c700 1 -- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec240ac450 msgr2=0x7fec2415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec29b9c700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec240ac450 0x7fec2415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec29b9c700 1 -- 172.21.15.84:0/1048704230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec20009500 con 0x7fec2415c750 2023-07-06T21:42:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec29b9c700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec24158950 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fec1c002230 tx=0x7fec1c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00aac0 con 0x7fec2415c750 2023-07-06T21:42:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c00ac40 con 0x7fec2415c750 2023-07-06T21:42:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec240c7630 con 0x7fec2415c750 2023-07-06T21:42:48.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c0070d0 con 0x7fec2415c750 2023-07-06T21:42:48.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.274+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec240c7bf0 con 0x7fec2415c750 2023-07-06T21:42:48.281 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.275+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec240c8110 con 0x7fec2415c750 2023-07-06T21:42:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.276+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fec1c007250 con 0x7fec2415c750 2023-07-06T21:42:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.278+0000 7fec19ffb700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0806ace0 0x7fec0806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.278+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec1c088f80 con 0x7fec2415c750 2023-07-06T21:42:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.278+0000 7fec2939b700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0806ace0 0x7fec0806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.279+0000 7fec2939b700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0806ace0 0x7fec0806d190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec20008070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:48.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.283+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fec240c8110 con 0x7fec2415c750 2023-07-06T21:42:48.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:42:48.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:42:48.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.487+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fec240c8110 con 0x7fec2415c750 2023-07-06T21:42:48.491 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.488+0000 7fec19ffb700 1 -- 172.21.15.84:0/1048704230 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fec240c8110 con 0x7fec2415c750 2023-07-06T21:42:48.491 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2023-07-06T21:42:48.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0806ace0 msgr2=0x7fec0806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.493 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0806ace0 0x7fec0806d190 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fec20000c00 tx=0x7fec20008070 comp rx=0 tx=0).stop 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 msgr2=0x7fec24158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec24158950 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fec1c002230 tx=0x7fec1c00cf50 comp rx=0 tx=0).stop 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 shutdown_connections 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fec0806ace0 0x7fec0806d190 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec2415c750 0x7fec24158950 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec240ac450 0x7fec2415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.494 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 --2- 172.21.15.84:0/1048704230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec240ad960 0x7fec24158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 >> 172.21.15.84:0/1048704230 conn(0x7fec2406b8c0 msgr2=0x7fec24159660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:48.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 shutdown_connections 2023-07-06T21:42:48.495 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.490+0000 7fec2b5ff700 1 -- 172.21.15.84:0/1048704230 wait complete. 2023-07-06T21:42:48.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-07-06T21:42:48.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.705+0000 7f0e9d7ff700 1 Processor -- start 2023-07-06T21:42:48.708 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.705+0000 7f0e9d7ff700 1 -- start start 2023-07-06T21:42:48.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.706+0000 7f0e9d7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0e980c90a0 0x7f0e98069e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.706+0000 7f0e9d7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9806cf40 con 0x7f0e9806a3d0 2023-07-06T21:42:48.714 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.706+0000 7f0e9d7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9806a7f0 con 0x7f0e9815c020 2023-07-06T21:42:48.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.706+0000 7f0e9d7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9815d400 con 0x7f0e980c90a0 2023-07-06T21:42:48.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.706+0000 7f0e977fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f0e9806a3d0 0x7f0e9806d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40696/0 (socket says 172.21.15.84:40696) 2023-07-06T21:42:48.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.706+0000 7f0e977fe700 1 -- 172.21.15.84:0/4068815956 learned_addr learned my addr 172.21.15.84:0/4068815956 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:48.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e967fc700 1 --2- 172.21.15.84:0/4068815956 >> v2:172.21.15.107:3300/0 conn(0x7f0e980c90a0 0x7f0e98069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287171584 0 0) 0x7f0e9806cf40 con 0x7f0e9806a3d0 2023-07-06T21:42:48.715 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e800057c0 con 0x7f0e9806a3d0 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171969720 0 0) 0x7f0e9806a7f0 con 0x7f0e9815c020 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e9806cf40 con 0x7f0e9815c020 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790420674 0 0) 0x7f0e800057c0 con 0x7f0e9806a3d0 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 >> v2:172.21.15.107:3300/0 conn(0x7f0e980c90a0 msgr2=0x7f0e98069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 --2- 172.21.15.84:0/4068815956 >> v2:172.21.15.107:3300/0 conn(0x7f0e980c90a0 0x7f0e98069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 >> v1:172.21.15.107:6789/0 conn(0x7f0e9815c020 legacy=0x7f0e9815c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.716 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9815cb70 con 0x7f0e9806a3d0 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f0e900019f0 con 0x7f0e9806a3d0 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 >> v1:172.21.15.84:6789/0 conn(0x7f0e9806a3d0 legacy=0x7f0e9806d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e80005da0 0x7f0e80006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e80006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e80007410 0x7f0e8000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8000e2f0 con 0x7f0e80006a60 2023-07-06T21:42:48.717 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8000e460 con 0x7f0e80005da0 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.707+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8000e620 con 0x7f0e80007410 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e80006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e96ffd700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e80005da0 0x7f0e80006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e977fe700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e80007410 0x7f0e8000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 -- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e80007410 msgr2=0x7f0e8000dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e80007410 0x7f0e8000dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 -- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e80005da0 msgr2=0x7f0e80006190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:48.718 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e80005da0 0x7f0e80006190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 -- 172.21.15.84:0/4068815956 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800057c0 con 0x7f0e80006a60 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e96ffd700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e80005da0 0x7f0e80006190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e977fe700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e80007410 0x7f0e8000dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e977fe700 1 -- 172.21.15.84:0/4068815956 reap_dead start 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e967fc700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e80006ed0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009b30 tx=0x7f0e80007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c848e88f3419d18 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00e6c0 con 0x7f0e80006a60 2023-07-06T21:42:48.719 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00e820 con 0x7f0e80006a60 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.708+0000 7f0e95ffb700 1 -- 172.21.15.84:0/4068815956 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00eb90 con 0x7f0e80006a60 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/4068815956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 msgr2=0x7f0e80006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e80006ed0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009b30 tx=0x7f0e80007a90 comp rx=0 tx=0).stop 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/4068815956 shutdown_connections 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/4068815956 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e80006ed0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/4068815956 >> 172.21.15.84:0/4068815956 conn(0x7f0e980cbf10 msgr2=0x7f0e980ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/4068815956 shutdown_connections 2023-07-06T21:42:48.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.709+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/4068815956 wait complete. 2023-07-06T21:42:48.721 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.711+0000 7f0e9d7ff700 1 Processor -- start 2023-07-06T21:42:48.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.711+0000 7f0e9d7ff700 1 -- start start 2023-07-06T21:42:48.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e9d7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e980c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.723 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e9d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e980ae710 0x7f0e98156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e9d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9815ace0 0x7f0e980b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e9d7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e98158af0 con 0x7f0e80006a60 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e9d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980ae500 con 0x7f0e980ae710 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e9d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e980c7ab0 con 0x7f0e9815ace0 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e96ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e980c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e96ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e980c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32896/0 (socket says 172.21.15.84:32896) 2023-07-06T21:42:48.724 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e96ffd700 1 -- 172.21.15.84:0/780075265 learned_addr learned my addr 172.21.15.84:0/780075265 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e977fe700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9815ace0 0x7f0e980b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.712+0000 7f0e967fc700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e980ae710 0x7f0e98156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e96ffd700 1 -- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9815ace0 msgr2=0x7f0e980b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e96ffd700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9815ace0 0x7f0e980b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e96ffd700 1 -- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e980ae710 msgr2=0x7f0e98156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e96ffd700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e980ae710 0x7f0e98156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.725 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e96ffd700 1 -- 172.21.15.84:0/780075265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980c7d90 con 0x7f0e80006a60 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e967fc700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e980ae710 0x7f0e98156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e977fe700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9815ace0 0x7f0e980b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e96ffd700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e980c73b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800fae0 tx=0x7f0e8800fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880052e0 con 0x7f0e80006a60 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e88005440 con 0x7f0e80006a60 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e98158430 con 0x7f0e80006a60 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e880038a0 con 0x7f0e80006a60 2023-07-06T21:42:48.726 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.713+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e981573d0 con 0x7f0e80006a60 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.714+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9804e1d0 con 0x7f0e80006a60 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.715+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0e88003a00 con 0x7f0e80006a60 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.716+0000 7f0e7f7fe700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0e8406ace0 0x7f0e8406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.716+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0e880893e0 con 0x7f0e80006a60 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.716+0000 7f0e967fc700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0e8406ace0 0x7f0e8406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.717+0000 7f0e967fc700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0e8406ace0 0x7f0e8406d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0e980c8070 tx=0x7f0e8c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.721+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0e880543f0 con 0x7f0e80006a60 2023-07-06T21:42:48.918 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.915+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0e980b4780 con 0x7f0e80006a60 2023-07-06T21:42:48.919 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.916+0000 7f0e7f7fe700 1 -- 172.21.15.84:0/780075265 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v53) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0e88003cb0 con 0x7f0e80006a60 2023-07-06T21:42:48.919 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 8 2023-07-06T21:42:48.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0e8406ace0 msgr2=0x7f0e8406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0e8406ace0 0x7f0e8406d190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0e980c8070 tx=0x7f0e8c008030 comp rx=0 tx=0).stop 2023-07-06T21:42:48.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 msgr2=0x7f0e980c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:48.921 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e980c73b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f0e8800fae0 tx=0x7f0e8800fea0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 shutdown_connections 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0e8406ace0 0x7f0e8406d190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e9815ace0 0x7f0e980b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e80006a60 0x7f0e980c73b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 --2- 172.21.15.84:0/780075265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e980ae710 0x7f0e98156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 >> 172.21.15.84:0/780075265 conn(0x7f0e980cbf10 msgr2=0x7f0e98070380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 shutdown_connections 2023-07-06T21:42:48.922 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:48.918+0000 7f0e9d7ff700 1 -- 172.21.15.84:0/780075265 wait complete. 2023-07-06T21:42:48.940 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-06T21:42:49.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.136+0000 7fdb04dff700 1 Processor -- start 2023-07-06T21:42:49.138 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.136+0000 7fdb04dff700 1 -- start start 2023-07-06T21:42:49.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdb04dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb00069e90 0x7fdb0006ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdb04dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb000b1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.140 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdb04dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb000c94b0 con 0x7fdb0006dde0 2023-07-06T21:42:49.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdb04dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0006d350 con 0x7fdb00069e90 2023-07-06T21:42:49.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdb04dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb000ca9b0 con 0x7fdb000cf280 2023-07-06T21:42:49.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdafe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb00069e90 0x7fdb0006ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdafe59c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdb00069e90 0x7fdb0006ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32902/0 (socket says 172.21.15.84:32902) 2023-07-06T21:42:49.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdafe59c700 1 -- 172.21.15.84:0/4244708734 learned_addr learned my addr 172.21.15.84:0/4244708734 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:49.141 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.137+0000 7fdafed9d700 1 --2- 172.21.15.84:0/4244708734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb000b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafe59c700 1 -- 172.21.15.84:0/4244708734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 msgr2=0x7fdb000b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:49.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafe59c700 1 --2- 172.21.15.84:0/4244708734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb000b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafe59c700 1 -- 172.21.15.84:0/4244708734 >> v1:172.21.15.107:6789/0 conn(0x7fdb0006dde0 legacy=0x7fdb0006a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafe59c700 1 -- 172.21.15.84:0/4244708734 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000cac90 con 0x7fdb00069e90 2023-07-06T21:42:49.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafe59c700 1 --2- 172.21.15.84:0/4244708734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb0006ce10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00098f0 tx=0x7fdaf0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725b34783a3891e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:49.142 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafdd9b700 1 -- 172.21.15.84:0/4244708734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000b8a0 con 0x7fdb00069e90 2023-07-06T21:42:49.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafdd9b700 1 -- 172.21.15.84:0/4244708734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf000ba00 con 0x7fdb00069e90 2023-07-06T21:42:49.143 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.138+0000 7fdafdd9b700 1 -- 172.21.15.84:0/4244708734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000bd90 con 0x7fdb00069e90 2023-07-06T21:42:49.145 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 -- 172.21.15.84:0/4244708734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 msgr2=0x7fdb0006ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 --2- 172.21.15.84:0/4244708734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb0006ce10 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00098f0 tx=0x7fdaf0009c00 comp rx=0 tx=0).stop 2023-07-06T21:42:49.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 -- 172.21.15.84:0/4244708734 shutdown_connections 2023-07-06T21:42:49.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 --2- 172.21.15.84:0/4244708734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb000b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.146 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 --2- 172.21.15.84:0/4244708734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb0006ce10 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 -- 172.21.15.84:0/4244708734 >> 172.21.15.84:0/4244708734 conn(0x7fdb000cbf10 msgr2=0x7fdb000ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:49.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 -- 172.21.15.84:0/4244708734 shutdown_connections 2023-07-06T21:42:49.147 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.139+0000 7fdb04dff700 1 -- 172.21.15.84:0/4244708734 wait complete. 2023-07-06T21:42:49.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.141+0000 7fdb04dff700 1 Processor -- start 2023-07-06T21:42:49.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.141+0000 7fdb04dff700 1 -- start start 2023-07-06T21:42:49.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdb04dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb000c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.148 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdb04dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb0006dde0 0x7fdb0015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdb04dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb0015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdb04dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0015a9f0 con 0x7fdb00069e90 2023-07-06T21:42:49.149 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdb04dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0015ac60 con 0x7fdb0006dde0 2023-07-06T21:42:49.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdb04dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0015af40 con 0x7fdb000cf280 2023-07-06T21:42:49.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb000c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb000c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32908/0 (socket says 172.21.15.84:32908) 2023-07-06T21:42:49.150 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 -- 172.21.15.84:0/1193400271 learned_addr learned my addr 172.21.15.84:0/1193400271 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:49.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 -- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 msgr2=0x7fdb0015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb0015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 -- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb0006dde0 msgr2=0x7fdb0015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdaf7fff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb0006dde0 0x7fdb0015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb0006dde0 0x7fdb0015cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdafe59c700 1 -- 172.21.15.84:0/1193400271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009550 con 0x7fdb00069e90 2023-07-06T21:42:49.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.142+0000 7fdaf7fff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb0006dde0 0x7fdb0015cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:49.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.143+0000 7fdafe59c700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb000c82b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009c40 tx=0x7fdaf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:49.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.143+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00036a0 con 0x7fdb00069e90 2023-07-06T21:42:49.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.143+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf001e070 con 0x7fdb00069e90 2023-07-06T21:42:49.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.143+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0015b220 con 0x7fdb00069e90 2023-07-06T21:42:49.153 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.143+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003a40 con 0x7fdb00069e90 2023-07-06T21:42:49.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.143+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb001f0e40 con 0x7fdb00069e90 2023-07-06T21:42:49.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.144+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0004e1d0 con 0x7fdb00069e90 2023-07-06T21:42:49.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.145+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdaf0003ba0 con 0x7fdb00069e90 2023-07-06T21:42:49.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.146+0000 7fdaf6ffd700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdae006ace0 0x7fdae006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.146+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdaf0020070 con 0x7fdb00069e90 2023-07-06T21:42:49.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.146+0000 7fdaf7fff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdae006ace0 0x7fdae006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.147+0000 7fdaf7fff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdae006ace0 0x7fdae006d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdb00152630 tx=0x7fdae8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:49.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.150+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdaf0055af0 con 0x7fdb00069e90 2023-07-06T21:42:49.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.346+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb000c97e0 con 0x7fdb00069e90 2023-07-06T21:42:49.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.346+0000 7fdaf6ffd700 1 -- 172.21.15.84:0/1193400271 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v53) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdaf0059110 con 0x7fdb00069e90 2023-07-06T21:42:49.349 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2023-07-06T21:42:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdae006ace0 msgr2=0x7fdae006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdae006ace0 0x7fdae006d190 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdb00152630 tx=0x7fdae8009040 comp rx=0 tx=0).stop 2023-07-06T21:42:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 msgr2=0x7fdb000c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb000c82b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009c40 tx=0x7fdaf0004750 comp rx=0 tx=0).stop 2023-07-06T21:42:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 shutdown_connections 2023-07-06T21:42:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdae006ace0 0x7fdae006d190 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb000cf280 0x7fdb0015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb00069e90 0x7fdb000c82b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 --2- 172.21.15.84:0/1193400271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb0006dde0 0x7fdb0015cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 >> 172.21.15.84:0/1193400271 conn(0x7fdb000cbf10 msgr2=0x7fdb000b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:49.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 shutdown_connections 2023-07-06T21:42:49.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.349+0000 7fdb04dff700 1 -- 172.21.15.84:0/1193400271 wait complete. 2023-07-06T21:42:49.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-06T21:42:49.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.564+0000 7f57fdbff700 1 Processor -- start 2023-07-06T21:42:49.567 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.565+0000 7f57fdbff700 1 -- start start 2023-07-06T21:42:49.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57fdbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b1a80 0x7f57f80d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57fdbff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80d1830 con 0x7f57f80b81c0 2023-07-06T21:42:49.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57fdbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80cf330 con 0x7f57f80cfd10 2023-07-06T21:42:49.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57fdbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80d1270 con 0x7f57f80b1a80 2023-07-06T21:42:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57f77fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f57f80b81c0 0x7f57f80cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40712/0 (socket says 172.21.15.84:40712) 2023-07-06T21:42:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57f77fe700 1 -- 172.21.15.84:0/3401899978 learned_addr learned my addr 172.21.15.84:0/3401899978 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:49.569 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.566+0000 7f57f7fff700 1 --2- 172.21.15.84:0/3401899978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b1a80 0x7f57f80d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620673455 0 0) 0x7f57f80d1830 con 0x7f57f80b81c0 2023-07-06T21:42:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e40035b0 con 0x7f57f80b81c0 2023-07-06T21:42:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042805015 0 0) 0x7f57f80cf330 con 0x7f57f80cfd10 2023-07-06T21:42:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57f80d1830 con 0x7f57f80cfd10 2023-07-06T21:42:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f7fff700 1 -- 172.21.15.84:0/3401899978 >> v1:172.21.15.107:6789/0 conn(0x7f57f80cfd10 legacy=0x7f57f80d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f7fff700 1 -- 172.21.15.84:0/3401899978 >> v1:172.21.15.84:6789/0 conn(0x7f57f80b81c0 legacy=0x7f57f80cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f7fff700 1 -- 172.21.15.84:0/3401899978 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80d1550 con 0x7f57f80b1a80 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f7fff700 1 --2- 172.21.15.84:0/3401899978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b1a80 0x7f57f80d2a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009dc0 tx=0x7f57f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bb72d53df43440 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f000e070 con 0x7f57f80b1a80 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f000bb20 con 0x7f57f80b1a80 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.567+0000 7f57f6ffd700 1 -- 172.21.15.84:0/3401899978 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0012430 con 0x7f57f80b1a80 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 -- 172.21.15.84:0/3401899978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b1a80 msgr2=0x7f57f80d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3401899978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b1a80 0x7f57f80d2a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009dc0 tx=0x7f57f000b330 comp rx=0 tx=0).stop 2023-07-06T21:42:49.573 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 -- 172.21.15.84:0/3401899978 shutdown_connections 2023-07-06T21:42:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3401899978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b1a80 0x7f57f80d2a90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 -- 172.21.15.84:0/3401899978 >> 172.21.15.84:0/3401899978 conn(0x7f57f806b2b0 msgr2=0x7f57f806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 -- 172.21.15.84:0/3401899978 shutdown_connections 2023-07-06T21:42:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.568+0000 7f57fdbff700 1 -- 172.21.15.84:0/3401899978 wait complete. 2023-07-06T21:42:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.570+0000 7f57fdbff700 1 Processor -- start 2023-07-06T21:42:49.574 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.570+0000 7f57fdbff700 1 -- start start 2023-07-06T21:42:49.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57fdbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57f80b1a80 0x7f57f80b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.576 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57fdbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b81c0 0x7f57f815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57fdbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 0x7f57f815f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57fdbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f815ca20 con 0x7f57f80cfd10 2023-07-06T21:42:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57fdbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f815ccc0 con 0x7f57f80b1a80 2023-07-06T21:42:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57fdbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f815cfa0 con 0x7f57f80b81c0 2023-07-06T21:42:49.577 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 0x7f57f815f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 0x7f57f815f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32922/0 (socket says 172.21.15.84:32922) 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 -- 172.21.15.84:0/3507436808 learned_addr learned my addr 172.21.15.84:0/3507436808 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f77fe700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57f80b1a80 0x7f57f80b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57effff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b81c0 0x7f57f815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 -- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b81c0 msgr2=0x7f57f815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b81c0 0x7f57f815ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.578 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 -- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57f80b1a80 msgr2=0x7f57f80b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57f80b1a80 0x7f57f80b72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.571+0000 7f57f7fff700 1 -- 172.21.15.84:0/3507436808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009a70 con 0x7f57f80cfd10 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57f77fe700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57f80b1a80 0x7f57f80b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57effff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b81c0 0x7f57f815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57f7fff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 0x7f57f815f3a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f57f0014040 tx=0x7f57f0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f001b810 con 0x7f57f80cfd10 2023-07-06T21:42:49.579 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f0020070 con 0x7f57f80cfd10 2023-07-06T21:42:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f001bbd0 con 0x7f57f80cfd10 2023-07-06T21:42:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f815d280 con 0x7f57f80cfd10 2023-07-06T21:42:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.572+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f815d840 con 0x7f57f80cfd10 2023-07-06T21:42:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.573+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f804e1d0 con 0x7f57f80cfd10 2023-07-06T21:42:49.580 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.574+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f57f001bd30 con 0x7f57f80cfd10 2023-07-06T21:42:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.575+0000 7f57f4ff9700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f57d806ace0 0x7f57d806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.575+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57f0022070 con 0x7f57f80cfd10 2023-07-06T21:42:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.575+0000 7f57f77fe700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f57d806ace0 0x7f57d806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.581 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.576+0000 7f57f77fe700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f57d806ace0 0x7f57d806d190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000f60 tx=0x7f57e8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:49.582 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.579+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f57f0056960 con 0x7f57f80cfd10 2023-07-06T21:42:49.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.778+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57f80c9a80 con 0x7f57f80cfd10 2023-07-06T21:42:49.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.779+0000 7f57f4ff9700 1 -- 172.21.15.84:0/3507436808 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v53) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57f0059f80 con 0x7f57f80cfd10 2023-07-06T21:42:49.782 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2023-07-06T21:42:49.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.781+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f57d806ace0 msgr2=0x7f57d806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.781+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f57d806ace0 0x7f57d806d190 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f57e8000f60 tx=0x7f57e8009040 comp rx=0 tx=0).stop 2023-07-06T21:42:49.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.781+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 msgr2=0x7f57f815f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:49.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.781+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 0x7f57f815f3a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f57f0014040 tx=0x7f57f0004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 shutdown_connections 2023-07-06T21:42:49.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f57d806ace0 0x7f57d806d190 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f57f80cfd10 0x7f57f815f3a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57f80b1a80 0x7f57f80b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 --2- 172.21.15.84:0/3507436808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57f80b81c0 0x7f57f815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:49.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 >> 172.21.15.84:0/3507436808 conn(0x7f57f806b2b0 msgr2=0x7f57f80d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:49.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 shutdown_connections 2023-07-06T21:42:49.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.782+0000 7f57fdbff700 1 -- 172.21.15.84:0/3507436808 wait complete. 2023-07-06T21:42:49.802 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:49.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.993+0000 7fe4063ff700 1 Processor -- start 2023-07-06T21:42:49.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.993+0000 7fe4063ff700 1 -- start start 2023-07-06T21:42:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe4063ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe4000b1f40 0x7fe4000b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe4063ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe4000cf520 0x7fe4000cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe4063ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4000cd820 con 0x7fe4000ced20 2023-07-06T21:42:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe4063ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000b2310 con 0x7fe4000b1f40 2023-07-06T21:42:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe4063ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4000cf190 con 0x7fe4000cf520 2023-07-06T21:42:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe4000b1f40 0x7fe4000b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fe4000b1f40 0x7fe4000b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32936/0 (socket says 172.21.15.84:32936) 2023-07-06T21:42:49.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 -- 172.21.15.84:0/4159037376 learned_addr learned my addr 172.21.15.84:0/4159037376 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe40499c700 1 --2- 172.21.15.84:0/4159037376 >> v2:172.21.15.107:3300/0 conn(0x7fe4000cf520 0x7fe4000cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 -- 172.21.15.84:0/4159037376 >> v2:172.21.15.107:3300/0 conn(0x7fe4000cf520 msgr2=0x7fe4000cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:49.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 --2- 172.21.15.84:0/4159037376 >> v2:172.21.15.107:3300/0 conn(0x7fe4000cf520 0x7fe4000cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 -- 172.21.15.84:0/4159037376 >> v1:172.21.15.107:6789/0 conn(0x7fe4000ced20 legacy=0x7fe4000d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.995+0000 7fe3fffff700 1 -- 172.21.15.84:0/4159037376 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000d2710 con 0x7fe4000b1f40 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.996+0000 7fe3fffff700 1 --2- 172.21.15.84:0/4159037376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4000b59c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe4000d3b30 tx=0x7fe3f0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81661b49277c8e60 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.996+0000 7fe3ff7fe700 1 -- 172.21.15.84:0/4159037376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0004230 con 0x7fe4000b1f40 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.996+0000 7fe3ff7fe700 1 -- 172.21.15.84:0/4159037376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0004390 con 0x7fe4000b1f40 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.996+0000 7fe3ff7fe700 1 -- 172.21.15.84:0/4159037376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f00036a0 con 0x7fe4000b1f40 2023-07-06T21:42:50.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.996+0000 7fe4063ff700 1 -- 172.21.15.84:0/4159037376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 msgr2=0x7fe4000b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.996+0000 7fe4063ff700 1 --2- 172.21.15.84:0/4159037376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4000b59c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe4000d3b30 tx=0x7fe3f0009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.997+0000 7fe4063ff700 1 -- 172.21.15.84:0/4159037376 shutdown_connections 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.997+0000 7fe4063ff700 1 --2- 172.21.15.84:0/4159037376 >> v2:172.21.15.107:3300/0 conn(0x7fe4000cf520 0x7fe4000cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.997+0000 7fe4063ff700 1 --2- 172.21.15.84:0/4159037376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4000b59c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.997+0000 7fe4063ff700 1 -- 172.21.15.84:0/4159037376 >> 172.21.15.84:0/4159037376 conn(0x7fe4000715c0 msgr2=0x7fe400071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.997+0000 7fe4063ff700 1 -- 172.21.15.84:0/4159037376 shutdown_connections 2023-07-06T21:42:50.001 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.997+0000 7fe4063ff700 1 -- 172.21.15.84:0/4159037376 wait complete. 2023-07-06T21:42:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.998+0000 7fe4063ff700 1 Processor -- start 2023-07-06T21:42:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.999+0000 7fe4063ff700 1 -- start start 2023-07-06T21:42:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.999+0000 7fe4063ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4001522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:49.999+0000 7fe4063ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4000ced20 0x7fe400158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe4063ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4000cf520 0x7fe4001588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.003 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe4063ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400158e00 con 0x7fe4000b1f40 2023-07-06T21:42:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe4063ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000cb840 con 0x7fe4000cf520 2023-07-06T21:42:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe4063ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000cbb20 con 0x7fe4000ced20 2023-07-06T21:42:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4001522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4001522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32944/0 (socket says 172.21.15.84:32944) 2023-07-06T21:42:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 -- 172.21.15.84:0/910490986 learned_addr learned my addr 172.21.15.84:0/910490986 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:50.004 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 -- 172.21.15.84:0/910490986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4000ced20 msgr2=0x7fe400158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4000ced20 0x7fe400158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 -- 172.21.15.84:0/910490986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4000cf520 msgr2=0x7fe4001588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4000cf520 0x7fe4001588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 -- 172.21.15.84:0/910490986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009550 con 0x7fe4000b1f40 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.000+0000 7fe3fffff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4001522b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009130 tx=0x7fe3f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.001+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0011630 con 0x7fe4000b1f40 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.001+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0011790 con 0x7fe4000b1f40 2023-07-06T21:42:50.005 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.001+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000cbe00 con 0x7fe4000b1f40 2023-07-06T21:42:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.001+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0011b40 con 0x7fe4000b1f40 2023-07-06T21:42:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.001+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000cc3c0 con 0x7fe4000b1f40 2023-07-06T21:42:50.006 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.002+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40004e1d0 con 0x7fe4000b1f40 2023-07-06T21:42:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.003+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fe3f0011ca0 con 0x7fe4000b1f40 2023-07-06T21:42:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.004+0000 7fe3fd7fa700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe3e006ace0 0x7fe3e006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.004+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe3f008ab30 con 0x7fe4000b1f40 2023-07-06T21:42:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.004+0000 7fe3f7fff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe3e006ace0 0x7fe3e006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.004+0000 7fe3f7fff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe3e006ace0 0x7fe3e006d190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe4000ccd40 tx=0x7fe3e8007a40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.009 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.008+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fe3f0055c80 con 0x7fe4000b1f40 2023-07-06T21:42:50.252 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.250+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4000b4f20 con 0x7fe4000b1f40 2023-07-06T21:42:50.253 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.251+0000 7fe3fd7fa700 1 -- 172.21.15.84:0/910490986 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4674 (secure 0 0 0) 0x7fe3f00592a0 con 0x7fe4000b1f40 2023-07-06T21:42:50.253 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:50.254 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:42:10.578278+0000","modified":"2023-07-06T21:42:13.588255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6834/2341549811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2341549811},{"type":"v1","addr":"172.21.15.107:6834","nonce":2341549811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T21:42:50.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe3e006ace0 msgr2=0x7fe3e006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.255 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe3e006ace0 0x7fe3e006d190 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe4000ccd40 tx=0x7fe3e8007a40 comp rx=0 tx=0).stop 2023-07-06T21:42:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 msgr2=0x7fe4001522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4001522b0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009130 tx=0x7fe3f0004750 comp rx=0 tx=0).stop 2023-07-06T21:42:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 shutdown_connections 2023-07-06T21:42:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fe3e006ace0 0x7fe3e006d190 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4000cf520 0x7fe4001588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4000b1f40 0x7fe4001522b0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 --2- 172.21.15.84:0/910490986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4000ced20 0x7fe400158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 >> 172.21.15.84:0/910490986 conn(0x7fe4000715c0 msgr2=0x7fe40006bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:50.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 shutdown_connections 2023-07-06T21:42:50.257 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.253+0000 7fe4063ff700 1 -- 172.21.15.84:0/910490986 wait complete. 2023-07-06T21:42:50.257 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-07-06T21:42:50.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:50.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.464+0000 7feee1dff700 1 Processor -- start 2023-07-06T21:42:50.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.464+0000 7feee1dff700 1 -- start start 2023-07-06T21:42:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feee1dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feedc0b83c0 0x7feedc0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feee1dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7feedc0d3710 0x7feedc0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feee1dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc0d0a80 con 0x7feedc0cfa90 2023-07-06T21:42:50.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feee1dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0cff10 con 0x7feedc0b83c0 2023-07-06T21:42:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feee1dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc0b8790 con 0x7feedc0d3710 2023-07-06T21:42:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feedb7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feedc0b83c0 0x7feedc0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feedb7fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feedc0b83c0 0x7feedc0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32960/0 (socket says 172.21.15.84:32960) 2023-07-06T21:42:50.468 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feedb7fe700 1 -- 172.21.15.84:0/3789447179 learned_addr learned my addr 172.21.15.84:0/3789447179 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:50.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.465+0000 7feed3fff700 1 --2- 172.21.15.84:0/3789447179 >> v2:172.21.15.107:3300/0 conn(0x7feedc0d3710 0x7feedc0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedb7fe700 1 -- 172.21.15.84:0/3789447179 >> v2:172.21.15.107:3300/0 conn(0x7feedc0d3710 msgr2=0x7feedc0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedb7fe700 1 --2- 172.21.15.84:0/3789447179 >> v2:172.21.15.107:3300/0 conn(0x7feedc0d3710 0x7feedc0cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedb7fe700 1 -- 172.21.15.84:0/3789447179 >> v1:172.21.15.107:6789/0 conn(0x7feedc0cfa90 legacy=0x7feedc15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedb7fe700 1 -- 172.21.15.84:0/3789447179 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0b4e70 con 0x7feedc0b83c0 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedb7fe700 1 --2- 172.21.15.84:0/3789447179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0b83c0 0x7feedc0cd200 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feedc0b8cb0 tx=0x7feecc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59eac6e9a1a8a079 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedaffd700 1 -- 172.21.15.84:0/3789447179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00b870 con 0x7feedc0b83c0 2023-07-06T21:42:50.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedaffd700 1 -- 172.21.15.84:0/3789447179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00b9d0 con 0x7feedc0b83c0 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.466+0000 7feedaffd700 1 -- 172.21.15.84:0/3789447179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00bd60 con 0x7feedc0b83c0 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 -- 172.21.15.84:0/3789447179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0b83c0 msgr2=0x7feedc0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 --2- 172.21.15.84:0/3789447179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0b83c0 0x7feedc0cd200 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feedc0b8cb0 tx=0x7feecc009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 -- 172.21.15.84:0/3789447179 shutdown_connections 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 --2- 172.21.15.84:0/3789447179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0b83c0 0x7feedc0cd200 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 --2- 172.21.15.84:0/3789447179 >> v2:172.21.15.107:3300/0 conn(0x7feedc0d3710 0x7feedc0cf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 -- 172.21.15.84:0/3789447179 >> 172.21.15.84:0/3789447179 conn(0x7feedc06b350 msgr2=0x7feedc06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:50.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 -- 172.21.15.84:0/3789447179 shutdown_connections 2023-07-06T21:42:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.467+0000 7feee1dff700 1 -- 172.21.15.84:0/3789447179 wait complete. 2023-07-06T21:42:50.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.469+0000 7feee1dff700 1 Processor -- start 2023-07-06T21:42:50.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.469+0000 7feee1dff700 1 -- start start 2023-07-06T21:42:50.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feee1dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feedc0b83c0 0x7feedc0c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feee1dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 0x7feedc152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feee1dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feedc0d3710 0x7feedc15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feee1dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c9550 con 0x7feedc0cfa90 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feee1dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c8210 con 0x7feedc0b83c0 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feee1dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0c84f0 con 0x7feedc0d3710 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 0x7feedc152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 0x7feedc152fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32962/0 (socket says 172.21.15.84:32962) 2023-07-06T21:42:50.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 -- 172.21.15.84:0/778546123 learned_addr learned my addr 172.21.15.84:0/778546123 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 -- 172.21.15.84:0/778546123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feedc0d3710 msgr2=0x7feedc15caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feedc0d3710 0x7feedc15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 -- 172.21.15.84:0/778546123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feedc0b83c0 msgr2=0x7feedc0c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feedc0b83c0 0x7feedc0c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 -- 172.21.15.84:0/778546123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feecc009550 con 0x7feedc0cfa90 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.470+0000 7feed3fff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 0x7feedc152fe0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feec400d7d0 tx=0x7feec400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.471+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40125f0 con 0x7feedc0cfa90 2023-07-06T21:42:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.471+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4012790 con 0x7feedc0cfa90 2023-07-06T21:42:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.471+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4012b40 con 0x7feedc0cfa90 2023-07-06T21:42:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.471+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc0c87d0 con 0x7feedc0cfa90 2023-07-06T21:42:50.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.471+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc1e6a80 con 0x7feedc0cfa90 2023-07-06T21:42:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.472+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc1e6ca0 con 0x7feedc0cfa90 2023-07-06T21:42:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.473+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feec4012ca0 con 0x7feedc0cfa90 2023-07-06T21:42:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.473+0000 7feed8ff9700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feebc06ace0 0x7feebc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.474+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feec4089890 con 0x7feedc0cfa90 2023-07-06T21:42:50.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.474+0000 7feedb7fe700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feebc06ace0 0x7feebc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.474+0000 7feedb7fe700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feebc06ace0 0x7feebc06d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feedc073a30 tx=0x7feecc010000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.478+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7feec4054930 con 0x7feedc0cfa90 2023-07-06T21:42:50.679 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.677+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feedc073550 con 0x7feedc0cfa90 2023-07-06T21:42:50.680 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.679+0000 7feed8ff9700 1 -- 172.21.15.84:0/778546123 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7feec4057f50 con 0x7feedc0cfa90 2023-07-06T21:42:50.681 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:50.681 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:50.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feebc06ace0 msgr2=0x7feebc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.683 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feebc06ace0 0x7feebc06d190 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feedc073a30 tx=0x7feecc010000 comp rx=0 tx=0).stop 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 msgr2=0x7feedc152fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 0x7feedc152fe0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7feec400d7d0 tx=0x7feec400db90 comp rx=0 tx=0).stop 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 shutdown_connections 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feebc06ace0 0x7feebc06d190 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feedc0d3710 0x7feedc15caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feedc0b83c0 0x7feedc0c9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.684 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 --2- 172.21.15.84:0/778546123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feedc0cfa90 0x7feedc152fe0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 >> 172.21.15.84:0/778546123 conn(0x7feedc06b350 msgr2=0x7feedc0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:50.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 shutdown_connections 2023-07-06T21:42:50.685 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.681+0000 7feee1dff700 1 -- 172.21.15.84:0/778546123 wait complete. 2023-07-06T21:42:50.702 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T21:42:50.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:50.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.894+0000 7f4af19ff700 1 Processor -- start 2023-07-06T21:42:50.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.894+0000 7f4af19ff700 1 -- start start 2023-07-06T21:42:50.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.895+0000 7f4af19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec159130 0x7f4aec0c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.895+0000 7f4af19ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4aec0c5000 0x7f4aec070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.898 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.895+0000 7f4af19ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec15d930 con 0x7f4aec070770 2023-07-06T21:42:50.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.895+0000 7f4af19ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec070b90 con 0x7f4aec0c5000 2023-07-06T21:42:50.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.895+0000 7f4af19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15cd20 con 0x7f4aec159130 2023-07-06T21:42:50.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeb7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f4aec070770 0x7f4aec0cc280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40722/0 (socket says 172.21.15.84:40722) 2023-07-06T21:42:50.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeb7fe700 1 -- 172.21.15.84:0/117930281 learned_addr learned my addr 172.21.15.84:0/117930281 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:50.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeaffd700 1 --2- 172.21.15.84:0/117930281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec159130 0x7f4aec0c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aea7fc700 1 --2- 172.21.15.84:0/117930281 >> v2:172.21.15.107:3300/0 conn(0x7f4aec0c5000 0x7f4aec070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4ae9ffb700 1 -- 172.21.15.84:0/117930281 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154684157 0 0) 0x7f4aec15d930 con 0x7f4aec070770 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4ae9ffb700 1 -- 172.21.15.84:0/117930281 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ad40035b0 con 0x7f4aec070770 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeaffd700 1 -- 172.21.15.84:0/117930281 >> v2:172.21.15.107:3300/0 conn(0x7f4aec0c5000 msgr2=0x7f4aec070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeaffd700 1 --2- 172.21.15.84:0/117930281 >> v2:172.21.15.107:3300/0 conn(0x7f4aec0c5000 0x7f4aec070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeaffd700 1 -- 172.21.15.84:0/117930281 >> v1:172.21.15.84:6789/0 conn(0x7f4aec070770 legacy=0x7f4aec0cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.896+0000 7f4aeaffd700 1 -- 172.21.15.84:0/117930281 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec15d000 con 0x7f4aec159130 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.897+0000 7f4aeaffd700 1 --2- 172.21.15.84:0/117930281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec159130 0x7f4aec0c4ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009840 tx=0x7f4adc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6671d1a65b303d4f server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.897+0000 7f4ae9ffb700 1 -- 172.21.15.84:0/117930281 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc011830 con 0x7f4aec159130 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.897+0000 7f4ae9ffb700 1 -- 172.21.15.84:0/117930281 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc011990 con 0x7f4aec159130 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.897+0000 7f4ae9ffb700 1 -- 172.21.15.84:0/117930281 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc011d00 con 0x7f4aec159130 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.897+0000 7f4af19ff700 1 -- 172.21.15.84:0/117930281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec159130 msgr2=0x7f4aec0c4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.897+0000 7f4af19ff700 1 --2- 172.21.15.84:0/117930281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec159130 0x7f4aec0c4ac0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009840 tx=0x7f4adc009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:50.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.898+0000 7f4af19ff700 1 -- 172.21.15.84:0/117930281 shutdown_connections 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.898+0000 7f4af19ff700 1 --2- 172.21.15.84:0/117930281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec159130 0x7f4aec0c4ac0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.898+0000 7f4af19ff700 1 --2- 172.21.15.84:0/117930281 >> v2:172.21.15.107:3300/0 conn(0x7f4aec0c5000 0x7f4aec070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.898+0000 7f4af19ff700 1 -- 172.21.15.84:0/117930281 >> 172.21.15.84:0/117930281 conn(0x7f4aec0cf3d0 msgr2=0x7f4aec0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.898+0000 7f4af19ff700 1 -- 172.21.15.84:0/117930281 shutdown_connections 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.898+0000 7f4af19ff700 1 -- 172.21.15.84:0/117930281 wait complete. 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.899+0000 7f4af19ff700 1 Processor -- start 2023-07-06T21:42:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.899+0000 7f4af19ff700 1 -- start start 2023-07-06T21:42:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.900+0000 7f4af19ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 0x7f4aec0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.900+0000 7f4af19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec0c5000 0x7f4aec14d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.900+0000 7f4af19ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4aec159130 0x7f4aec146000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.900+0000 7f4af19ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec14ca70 con 0x7f4aec070770 2023-07-06T21:42:50.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.900+0000 7f4af19ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec1484b0 con 0x7f4aec159130 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.900+0000 7f4af19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec0c5f20 con 0x7f4aec0c5000 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 0x7f4aec0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 0x7f4aec0afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32968/0 (socket says 172.21.15.84:32968) 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 -- 172.21.15.84:0/988583719 learned_addr learned my addr 172.21.15.84:0/988583719 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aea7fc700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec0c5000 0x7f4aec14d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeb7fe700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4aec159130 0x7f4aec146000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.906 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 -- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec0c5000 msgr2=0x7f4aec14d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec0c5000 0x7f4aec14d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 -- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4aec159130 msgr2=0x7f4aec146000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4aec159130 0x7f4aec146000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 -- 172.21.15.84:0/988583719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4adc0094a0 con 0x7f4aec070770 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeb7fe700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4aec159130 0x7f4aec146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4aeaffd700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 0x7f4aec0afc00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009fd0 tx=0x7f4adc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.901+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0045d0 con 0x7f4aec070770 2023-07-06T21:42:50.907 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.902+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4adc01f070 con 0x7f4aec070770 2023-07-06T21:42:50.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.902+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec0b0140 con 0x7f4aec070770 2023-07-06T21:42:50.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.902+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4adc0038a0 con 0x7f4aec070770 2023-07-06T21:42:50.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.902+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec14c420 con 0x7f4aec070770 2023-07-06T21:42:50.908 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.903+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec14c280 con 0x7f4aec070770 2023-07-06T21:42:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.904+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f4adc003a00 con 0x7f4aec070770 2023-07-06T21:42:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.905+0000 7f4ad3fff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4ad806ace0 0x7f4ad806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.905+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4adc021070 con 0x7f4aec070770 2023-07-06T21:42:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.905+0000 7f4aea7fc700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4ad806ace0 0x7f4ad806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:50.909 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.905+0000 7f4aea7fc700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4ad806ace0 0x7f4ad806d190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0cd770 tx=0x7f4ae0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:50.910 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:50.909+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4aec14c280 con 0x7f4aec070770 2023-07-06T21:42:51.151 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.149+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4aec14c280 con 0x7f4aec070770 2023-07-06T21:42:51.152 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.150+0000 7f4ad3fff700 1 -- 172.21.15.84:0/988583719 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f4aec14c280 con 0x7f4aec070770 2023-07-06T21:42:51.152 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:51.152 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:42:10.578278+0000","modified":"2023-07-06T21:42:13.588255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6834/2341549811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2341549811},{"type":"v1","addr":"172.21.15.107:6834","nonce":2341549811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T21:42:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4ad806ace0 msgr2=0x7f4ad806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4ad806ace0 0x7f4ad806d190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0cd770 tx=0x7f4ae0008040 comp rx=0 tx=0).stop 2023-07-06T21:42:51.154 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 msgr2=0x7f4aec0afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 0x7f4aec0afc00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009fd0 tx=0x7f4adc004750 comp rx=0 tx=0).stop 2023-07-06T21:42:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 shutdown_connections 2023-07-06T21:42:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f4ad806ace0 0x7f4ad806d190 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4aec159130 0x7f4aec146000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4aec070770 0x7f4aec0afc00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.155 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 --2- 172.21.15.84:0/988583719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4aec0c5000 0x7f4aec14d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 >> 172.21.15.84:0/988583719 conn(0x7f4aec0cf3d0 msgr2=0x7f4aec06d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 shutdown_connections 2023-07-06T21:42:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.152+0000 7f4af19ff700 1 -- 172.21.15.84:0/988583719 wait complete. 2023-07-06T21:42:51.156 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-07-06T21:42:51.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:42:51.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.362+0000 7fa001dff700 1 Processor -- start 2023-07-06T21:42:51.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.362+0000 7fa001dff700 1 -- start start 2023-07-06T21:42:51.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.363+0000 7fa001dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 0x7f9ffc0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.363+0000 7fa001dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9ffc0d1580 0x7f9ffc15c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.363+0000 7fa001dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ffc0d0990 con 0x7f9ffc0d3580 2023-07-06T21:42:51.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.363+0000 7fa001dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ffc0d3af0 con 0x7f9ffc0d1580 2023-07-06T21:42:51.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.363+0000 7fa001dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cd9d0 con 0x7f9ffc0b8530 2023-07-06T21:42:51.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.364+0000 7f9ffb7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 0x7f9ffc0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.364+0000 7f9ffb7fe700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 0x7f9ffc0cee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.84:47728/0 (socket says 172.21.15.84:47728) 2023-07-06T21:42:51.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.364+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/4141245150 learned_addr learned my addr 172.21.15.84:0/4141245150 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:51.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.364+0000 7f9ffbfff700 1 --2- 172.21.15.84:0/4141245150 >> v2:172.21.15.107:3300/0 conn(0x7f9ffc0d1580 0x7f9ffc15c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.364+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/4141245150 >> v2:172.21.15.107:3300/0 conn(0x7f9ffc0d1580 msgr2=0x7f9ffc15c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:51.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.364+0000 7f9ffb7fe700 1 --2- 172.21.15.84:0/4141245150 >> v2:172.21.15.107:3300/0 conn(0x7f9ffc0d1580 0x7f9ffc15c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/4141245150 >> v1:172.21.15.107:6789/0 conn(0x7f9ffc0d3580 legacy=0x7f9ffc0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/4141245150 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ffc0cd410 con 0x7f9ffc0b8530 2023-07-06T21:42:51.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffaffd700 1 -- 172.21.15.84:0/4141245150 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512937294 0 0) 0x7f9ffc0d0990 con 0x7f9ffc0d3580 2023-07-06T21:42:51.368 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffb7fe700 1 --2- 172.21.15.84:0/4141245150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 0x7f9ffc0cee70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9fec006fd0 tx=0x7f9fec009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d998b97b0d12fe3e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffaffd700 1 -- 172.21.15.84:0/4141245150 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec005e30 con 0x7f9ffc0b8530 2023-07-06T21:42:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffaffd700 1 -- 172.21.15.84:0/4141245150 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec004030 con 0x7f9ffc0b8530 2023-07-06T21:42:51.369 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.365+0000 7f9ffaffd700 1 -- 172.21.15.84:0/4141245150 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0043e0 con 0x7f9ffc0b8530 2023-07-06T21:42:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 -- 172.21.15.84:0/4141245150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 msgr2=0x7f9ffc0cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 --2- 172.21.15.84:0/4141245150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 0x7f9ffc0cee70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9fec006fd0 tx=0x7f9fec009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 -- 172.21.15.84:0/4141245150 shutdown_connections 2023-07-06T21:42:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 --2- 172.21.15.84:0/4141245150 >> v2:172.21.15.107:3300/0 conn(0x7f9ffc0d1580 0x7f9ffc15c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.370 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 --2- 172.21.15.84:0/4141245150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0b8530 0x7f9ffc0cee70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 -- 172.21.15.84:0/4141245150 >> 172.21.15.84:0/4141245150 conn(0x7f9ffc0715c0 msgr2=0x7f9ffc071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 -- 172.21.15.84:0/4141245150 shutdown_connections 2023-07-06T21:42:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.366+0000 7fa001dff700 1 -- 172.21.15.84:0/4141245150 wait complete. 2023-07-06T21:42:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.367+0000 7fa001dff700 1 Processor -- start 2023-07-06T21:42:51.371 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.368+0000 7fa001dff700 1 -- start start 2023-07-06T21:42:51.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.368+0000 7fa001dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 0x7f9ffc1e4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7fa001dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ffc0d1580 0x7f9ffc146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7fa001dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0d3580 0x7f9ffc1473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7fa001dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc0cc640 con 0x7f9ffc0b8530 2023-07-06T21:42:51.373 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7fa001dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc147b80 con 0x7f9ffc0d1580 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7fa001dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ffc147e60 con 0x7f9ffc0d3580 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 0x7f9ffc1e4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 0x7f9ffc1e4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32972/0 (socket says 172.21.15.84:32972) 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/1690294782 learned_addr learned my addr 172.21.15.84:0/1690294782 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffbfff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0d3580 0x7f9ffc1473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ff3fff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ffc0d1580 0x7f9ffc146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.374 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0d3580 msgr2=0x7f9ffc1473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0d3580 0x7f9ffc1473d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ffc0d1580 msgr2=0x7f9ffc146e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ffc0d1580 0x7f9ffc146e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 -- 172.21.15.84:0/1690294782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009500 con 0x7f9ffc0b8530 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.369+0000 7f9ffb7fe700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 0x7f9ffc1e4ba0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009c10 tx=0x7f9fec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.370+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec003dd0 con 0x7f9ffc0b8530 2023-07-06T21:42:51.375 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.370+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec01c420 con 0x7f9ffc0b8530 2023-07-06T21:42:51.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.370+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ffc148140 con 0x7f9ffc0b8530 2023-07-06T21:42:51.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.370+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec01c7b0 con 0x7f9ffc0b8530 2023-07-06T21:42:51.376 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.370+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ffc0c7070 con 0x7f9ffc0b8530 2023-07-06T21:42:51.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.371+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ffc04e240 con 0x7f9ffc0b8530 2023-07-06T21:42:51.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.372+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9fec01c910 con 0x7f9ffc0b8530 2023-07-06T21:42:51.377 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.372+0000 7f9ff8ff9700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fdc06ace0 0x7f9fdc06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.373+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9fec08a5d0 con 0x7f9ffc0b8530 2023-07-06T21:42:51.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.373+0000 7f9ff3fff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fdc06ace0 0x7f9fdc06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.378 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.373+0000 7f9ff3fff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fdc06ace0 0x7f9fdc06d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc15b980 tx=0x7f9fe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:51.379 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.377+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9fec055670 con 0x7f9ffc0b8530 2023-07-06T21:42:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.617+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ffc0b2660 con 0x7f9ffc0b8530 2023-07-06T21:42:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.617+0000 7f9ff8ff9700 1 -- 172.21.15.84:0/1690294782 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4674 (secure 0 0 0) 0x7f9fec058c90 con 0x7f9ffc0b8530 2023-07-06T21:42:51.620 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:51.620 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-07-06T21:42:10.578278+0000","modified":"2023-07-06T21:42:13.588255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"d","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6834/2341549811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":2341549811},{"type":"v1","addr":"172.21.15.107:6834","nonce":2341549811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-07-06T21:42:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fdc06ace0 msgr2=0x7f9fdc06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fdc06ace0 0x7f9fdc06d190 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc15b980 tx=0x7f9fe4009040 comp rx=0 tx=0).stop 2023-07-06T21:42:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 msgr2=0x7f9ffc1e4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 0x7f9ffc1e4ba0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009c10 tx=0x7f9fec004750 comp rx=0 tx=0).stop 2023-07-06T21:42:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 shutdown_connections 2023-07-06T21:42:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f9fdc06ace0 0x7f9fdc06d190 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ffc0d3580 0x7f9ffc1473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9ffc0b8530 0x7f9ffc1e4ba0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 --2- 172.21.15.84:0/1690294782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ffc0d1580 0x7f9ffc146e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 >> 172.21.15.84:0/1690294782 conn(0x7f9ffc0715c0 msgr2=0x7f9ffc0cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:51.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 shutdown_connections 2023-07-06T21:42:51.624 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.620+0000 7fa001dff700 1 -- 172.21.15.84:0/1690294782 wait complete. 2023-07-06T21:42:51.624 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 24 2023-07-06T21:42:51.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:51.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.832+0000 7fed8d7ff700 1 Processor -- start 2023-07-06T21:42:51.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.833+0000 7fed8d7ff700 1 -- start start 2023-07-06T21:42:51.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed8d7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed880b83c0 0x7fed880cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed8d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880d3710 0x7fed880cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed8d7ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed880cfee0 con 0x7fed880cfa90 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed8d7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed880d0a80 con 0x7fed880b83c0 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed8d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880b8790 con 0x7fed880d3710 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed877fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fed880cfa90 0x7fed8815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40732/0 (socket says 172.21.15.84:40732) 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed877fe700 1 -- 172.21.15.84:0/2961499407 learned_addr learned my addr 172.21.15.84:0/2961499407 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed867fc700 1 --2- 172.21.15.84:0/2961499407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880d3710 0x7fed880cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed86ffd700 1 --2- 172.21.15.84:0/2961499407 >> v2:172.21.15.107:3300/0 conn(0x7fed880b83c0 0x7fed880cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed85ffb700 1 -- 172.21.15.84:0/2961499407 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175353028 0 0) 0x7fed80001490 con 0x7fed880cfa90 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.834+0000 7fed85ffb700 1 -- 172.21.15.84:0/2961499407 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed700035b0 con 0x7fed880cfa90 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed867fc700 1 -- 172.21.15.84:0/2961499407 >> v2:172.21.15.107:3300/0 conn(0x7fed880b83c0 msgr2=0x7fed880cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed85ffb700 1 -- 172.21.15.84:0/2961499407 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1160920028 0 0) 0x7fed700035b0 con 0x7fed880cfa90 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed867fc700 1 --2- 172.21.15.84:0/2961499407 >> v2:172.21.15.107:3300/0 conn(0x7fed880b83c0 0x7fed880cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed867fc700 1 -- 172.21.15.84:0/2961499407 >> v1:172.21.15.84:6789/0 conn(0x7fed880cfa90 legacy=0x7fed8815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed867fc700 1 -- 172.21.15.84:0/2961499407 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880b4e70 con 0x7fed880d3710 2023-07-06T21:42:51.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed86ffd700 1 --2- 172.21.15.84:0/2961499407 >> v2:172.21.15.107:3300/0 conn(0x7fed880b83c0 0x7fed880cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:51.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed867fc700 1 --2- 172.21.15.84:0/2961499407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880d3710 0x7fed880cf550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009840 tx=0x7fed7c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9612dfcf0ad632 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:51.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed85ffb700 1 -- 172.21.15.84:0/2961499407 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80001b80 con 0x7fed880d3710 2023-07-06T21:42:51.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed85ffb700 1 -- 172.21.15.84:0/2961499407 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c004030 con 0x7fed880d3710 2023-07-06T21:42:51.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.835+0000 7fed85ffb700 1 -- 172.21.15.84:0/2961499407 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0043a0 con 0x7fed880d3710 2023-07-06T21:42:51.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 -- 172.21.15.84:0/2961499407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880d3710 msgr2=0x7fed880cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/2961499407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880d3710 0x7fed880cf550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009840 tx=0x7fed7c009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:51.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 -- 172.21.15.84:0/2961499407 shutdown_connections 2023-07-06T21:42:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/2961499407 >> v2:172.21.15.107:3300/0 conn(0x7fed880b83c0 0x7fed880cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/2961499407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880d3710 0x7fed880cf550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 -- 172.21.15.84:0/2961499407 >> 172.21.15.84:0/2961499407 conn(0x7fed8806b350 msgr2=0x7fed8806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 -- 172.21.15.84:0/2961499407 shutdown_connections 2023-07-06T21:42:51.842 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.836+0000 7fed8d7ff700 1 -- 172.21.15.84:0/2961499407 wait complete. 2023-07-06T21:42:51.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.838+0000 7fed8d7ff700 1 Processor -- start 2023-07-06T21:42:51.843 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.838+0000 7fed8d7ff700 1 -- start start 2023-07-06T21:42:51.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed8d7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880b83c0 0x7fed8815db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed8d7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 0x7fed8815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.848 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed8d7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed880d3710 0x7fed880b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed8d7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880cbe50 con 0x7fed880cfa90 2023-07-06T21:42:51.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed8d7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed880cb240 con 0x7fed880d3710 2023-07-06T21:42:51.849 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed8d7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed880cb520 con 0x7fed880b83c0 2023-07-06T21:42:51.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 0x7fed8815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 0x7fed8815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32988/0 (socket says 172.21.15.84:32988) 2023-07-06T21:42:51.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 -- 172.21.15.84:0/585307425 learned_addr learned my addr 172.21.15.84:0/585307425 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:51.850 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed86ffd700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880b83c0 0x7fed8815db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed877fe700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed880d3710 0x7fed880b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 -- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880b83c0 msgr2=0x7fed8815db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880b83c0 0x7fed8815db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 -- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed880d3710 msgr2=0x7fed880b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:51.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed880d3710 0x7fed880b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:51.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 -- 172.21.15.84:0/585307425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed780094a0 con 0x7fed880cfa90 2023-07-06T21:42:51.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed877fe700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed880d3710 0x7fed880b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:51.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.839+0000 7fed867fc700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 0x7fed8815c930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0090b0 tx=0x7fed7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:51.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.840+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c003df0 con 0x7fed880cfa90 2023-07-06T21:42:51.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.840+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c01f070 con 0x7fed880cfa90 2023-07-06T21:42:51.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.840+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c011870 con 0x7fed880cfa90 2023-07-06T21:42:51.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.840+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c0094a0 con 0x7fed880cfa90 2023-07-06T21:42:51.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.840+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed880cbae0 con 0x7fed880cfa90 2023-07-06T21:42:51.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.841+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed880b1f70 con 0x7fed880cfa90 2023-07-06T21:42:51.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.842+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fed7c021030 con 0x7fed880cfa90 2023-07-06T21:42:51.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.843+0000 7fed6f7fe700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fed7406ace0 0x7fed7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:51.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.843+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fed7c08a800 con 0x7fed880cfa90 2023-07-06T21:42:51.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.843+0000 7fed86ffd700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fed7406ace0 0x7fed7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:51.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.844+0000 7fed86ffd700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fed7406ace0 0x7fed7406d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed78009c60 tx=0x7fed78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:51.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:51.847+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fed7c055940 con 0x7fed880cfa90 2023-07-06T21:42:52.048 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.046+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed880cbf90 con 0x7fed880cfa90 2023-07-06T21:42:52.049 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.047+0000 7fed6f7fe700 1 -- 172.21.15.84:0/585307425 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18347 (secure 0 0 0) 0x7fed7c058f60 con 0x7fed880cfa90 2023-07-06T21:42:52.050 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:52.050 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":53,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:11.580909+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-07-06T21:42:08.774391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-07-06T21:42:09.558402+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:52.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.049+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fed7406ace0 msgr2=0x7fed7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:52.052 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.049+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fed7406ace0 0x7fed7406d190 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed78009c60 tx=0x7fed78008040 comp rx=0 tx=0).stop 2023-07-06T21:42:52.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.049+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 msgr2=0x7fed8815c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:52.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.049+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 0x7fed8815c930 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fed7c0090b0 tx=0x7fed7c004750 comp rx=0 tx=0).stop 2023-07-06T21:42:52.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 shutdown_connections 2023-07-06T21:42:52.053 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fed7406ace0 0x7fed7406d190 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed880d3710 0x7fed880b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed880b83c0 0x7fed8815db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 --2- 172.21.15.84:0/585307425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed880cfa90 0x7fed8815c930 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 >> 172.21.15.84:0/585307425 conn(0x7fed8806b350 msgr2=0x7fed8815aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:52.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 shutdown_connections 2023-07-06T21:42:52.054 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.050+0000 7fed8d7ff700 1 -- 172.21.15.84:0/585307425 wait complete. 2023-07-06T21:42:52.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-07-06T21:42:52.267 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.265+0000 7fb9de5ff700 1 Processor -- start 2023-07-06T21:42:52.268 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.266+0000 7fb9de5ff700 1 -- start start 2023-07-06T21:42:52.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9de5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:52.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9de5ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9d80d3710 0x7fb9d80cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:52.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9de5ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d80cfee0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9de5ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d80d0a80 con 0x7fb9d80d3710 2023-07-06T21:42:52.269 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9de5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80b8790 con 0x7fb9d80b83c0 2023-07-06T21:42:52.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d77fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9d80d3710 0x7fb9d80cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:52.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9dcb9c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb9d80cfa90 0x7fb9d815feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:40738/0 (socket says 172.21.15.84:40738) 2023-07-06T21:42:52.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d77fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb9d80d3710 0x7fb9d80cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32990/0 (socket says 172.21.15.84:32990) 2023-07-06T21:42:52.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9dcb9c700 1 -- 172.21.15.84:0/2821912666 learned_addr learned my addr 172.21.15.84:0/2821912666 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:52.270 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d7fff700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:52.271 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d77fe700 1 -- 172.21.15.84:0/2821912666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 msgr2=0x7fb9d80cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:52.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d77fe700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d77fe700 1 -- 172.21.15.84:0/2821912666 >> v1:172.21.15.84:6789/0 conn(0x7fb9d80cfa90 legacy=0x7fb9d815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:52.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d77fe700 1 -- 172.21.15.84:0/2821912666 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80b4e70 con 0x7fb9d80d3710 2023-07-06T21:42:52.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.267+0000 7fb9d7fff700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:52.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.268+0000 7fb9d77fe700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80d3710 0x7fb9d80cf550 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009840 tx=0x7fb9cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2336b8d72e99572 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:52.272 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.268+0000 7fb9d6ffd700 1 -- 172.21.15.84:0/2821912666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc011830 con 0x7fb9d80d3710 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.268+0000 7fb9d6ffd700 1 -- 172.21.15.84:0/2821912666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc011990 con 0x7fb9d80d3710 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.268+0000 7fb9d6ffd700 1 -- 172.21.15.84:0/2821912666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc011d20 con 0x7fb9d80d3710 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.268+0000 7fb9de5ff700 1 -- 172.21.15.84:0/2821912666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80d3710 msgr2=0x7fb9d80cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.268+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80d3710 0x7fb9d80cf550 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009840 tx=0x7fb9cc009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.269+0000 7fb9de5ff700 1 -- 172.21.15.84:0/2821912666 shutdown_connections 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.269+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.269+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/2821912666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80d3710 0x7fb9d80cf550 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.273 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.269+0000 7fb9de5ff700 1 -- 172.21.15.84:0/2821912666 >> 172.21.15.84:0/2821912666 conn(0x7fb9d806b350 msgr2=0x7fb9d806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:52.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.269+0000 7fb9de5ff700 1 -- 172.21.15.84:0/2821912666 shutdown_connections 2023-07-06T21:42:52.274 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.269+0000 7fb9de5ff700 1 -- 172.21.15.84:0/2821912666 wait complete. 2023-07-06T21:42:52.275 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.271+0000 7fb9de5ff700 1 Processor -- start 2023-07-06T21:42:52.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.271+0000 7fb9de5ff700 1 -- start start 2023-07-06T21:42:52.276 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.272+0000 7fb9de5ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:52.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.272+0000 7fb9de5ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 0x7fb9d815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:52.282 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.272+0000 7fb9de5ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d80d3710 0x7fb9d80c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.272+0000 7fb9de5ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d815cad0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.272+0000 7fb9de5ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d80cbab0 con 0x7fb9d80d3710 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.272+0000 7fb9de5ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d815ed80 con 0x7fb9d80b83c0 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 0x7fb9d815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 0x7fb9d815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33002/0 (socket says 172.21.15.84:33002) 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 -- 172.21.15.84:0/1189402563 learned_addr learned my addr 172.21.15.84:0/1189402563 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d7fff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:52.283 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9dcb9c700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d80d3710 0x7fb9d80c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 -- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 msgr2=0x7fb9d80c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 -- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d80d3710 msgr2=0x7fb9d80c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d80d3710 0x7fb9d80c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 -- 172.21.15.84:0/1189402563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0094a0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9dcb9c700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d80d3710 0x7fb9d80c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:52.284 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d77fe700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 0x7fb9d815e840 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009b90 tx=0x7fb9cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.273+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc0036a0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.274+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc01e070 con 0x7fb9d80cfa90 2023-07-06T21:42:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.274+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc003a60 con 0x7fb9d80cfa90 2023-07-06T21:42:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.274+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d80c6bb0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.274+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d80ca5d0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.275+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d80d1620 con 0x7fb9d80cfa90 2023-07-06T21:42:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.276+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb9cc003bc0 con 0x7fb9d80cfa90 2023-07-06T21:42:52.286 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.276+0000 7fb9d4ff9700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb9c406ace0 0x7fb9c406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.277+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb9cc08a550 con 0x7fb9d80cfa90 2023-07-06T21:42:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.277+0000 7fb9d7fff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb9c406ace0 0x7fb9c406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.277+0000 7fb9d7fff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb9c406ace0 0x7fb9c406d190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:52.287 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.281+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fb9cc055630 con 0x7fb9d80cfa90 2023-07-06T21:42:52.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:52.521+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb9d80d1340 con 0x7fb9d80cfa90 2023-07-06T21:42:53.103 INFO:tasks.ceph.mon.a.smithi084.stderr:2023-07-06T21:42:53.101+0000 7feff05f9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-07-06T21:42:53.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.108+0000 7fb9d4ff9700 1 -- 172.21.15.84:0/1189402563 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb9cc058c50 con 0x7fb9d80cfa90 2023-07-06T21:42:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb9c406ace0 msgr2=0x7fb9c406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:53.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb9c406ace0 0x7fb9c406d190 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80098f0 tx=0x7fb9c8008040 comp rx=0 tx=0).stop 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 msgr2=0x7fb9d815e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 0x7fb9d815e840 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc009b90 tx=0x7fb9cc004750 comp rx=0 tx=0).stop 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 shutdown_connections 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb9c406ace0 0x7fb9c406d190 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9d80d3710 0x7fb9d80c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9d80b83c0 0x7fb9d80c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 --2- 172.21.15.84:0/1189402563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9d80cfa90 0x7fb9d815e840 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 >> 172.21.15.84:0/1189402563 conn(0x7fb9d806b350 msgr2=0x7fb9d80b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 shutdown_connections 2023-07-06T21:42:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.110+0000 7fb9de5ff700 1 -- 172.21.15.84:0/1189402563 wait complete. 2023-07-06T21:42:53.114 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-07-06T21:42:53.117 INFO:tasks.ceph.mds.d.smithi107.stderr: -2967> 2023-07-06T21:42:15.058+0000 7f208c1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-06T21:42:53.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-07-06T21:42:53.167 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2023-07-06T21:42:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.325+0000 7f0fb3bff700 1 Processor -- start 2023-07-06T21:42:53.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.325+0000 7f0fb3bff700 1 -- start start 2023-07-06T21:42:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb3bff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 0x7f0fac0cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb3bff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0fac0b1a80 0x7f0fac0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb3bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb3bff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac0d1830 con 0x7f0fac0b81c0 2023-07-06T21:42:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb3bff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac0cf330 con 0x7f0fac0b1a80 2023-07-06T21:42:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb3bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0d1270 con 0x7f0fac0ceda0 2023-07-06T21:42:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb199b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 0x7f0fac0cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb199b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 0x7f0fac0cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44356/0 (socket says 172.21.15.84:44356) 2023-07-06T21:42:53.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb119a700 1 --2- 172.21.15.84:0/3420888501 >> v2:172.21.15.107:3300/0 conn(0x7f0fac0b1a80 0x7f0fac0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.326+0000 7f0fb199b700 1 -- 172.21.15.84:0/3420888501 learned_addr learned my addr 172.21.15.84:0/3420888501 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:53.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb219c700 1 --2- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb119a700 1 -- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 msgr2=0x7f0fac15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb119a700 1 --2- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb119a700 1 -- 172.21.15.84:0/3420888501 >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 msgr2=0x7f0fac0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb119a700 1 --2- 172.21.15.84:0/3420888501 >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 0x7f0fac0cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb119a700 1 -- 172.21.15.84:0/3420888501 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac0d1550 con 0x7f0fac0b1a80 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb199b700 1 --2- 172.21.15.84:0/3420888501 >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 0x7f0fac0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb119a700 1 --2- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac0cfb40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0d0770 tx=0x7f0f9c009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce0e93db85e373cd server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb0999700 1 -- 172.21.15.84:0/3420888501 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c011830 con 0x7f0fac0b1a80 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb0999700 1 -- 172.21.15.84:0/3420888501 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c011990 con 0x7f0fac0b1a80 2023-07-06T21:42:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.327+0000 7f0fb0999700 1 -- 172.21.15.84:0/3420888501 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c011d20 con 0x7f0fac0b1a80 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 -- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 msgr2=0x7f0fac0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac0cfb40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0d0770 tx=0x7f0f9c009b20 comp rx=0 tx=0).stop 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 -- 172.21.15.84:0/3420888501 shutdown_connections 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/3420888501 >> v2:172.21.15.84:3300/0 conn(0x7f0fac0b81c0 0x7f0fac0cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/3420888501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac0cfb40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 -- 172.21.15.84:0/3420888501 >> 172.21.15.84:0/3420888501 conn(0x7f0fac06b2b0 msgr2=0x7f0fac06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 -- 172.21.15.84:0/3420888501 shutdown_connections 2023-07-06T21:42:53.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.328+0000 7f0fb3bff700 1 -- 172.21.15.84:0/3420888501 wait complete. 2023-07-06T21:42:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.330+0000 7f0fb3bff700 1 Processor -- start 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.331+0000 7f0fb3bff700 1 -- start start 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb3bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb3bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 0x7f0fac15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb3bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac1f1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb3bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0c7ea0 con 0x7f0fac0b81c0 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb3bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0c8140 con 0x7f0fac0b1a80 2023-07-06T21:42:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb3bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac0c8420 con 0x7f0fac0ceda0 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb119a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 0x7f0fac15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb119a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 0x7f0fac15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44358/0 (socket says 172.21.15.84:44358) 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb119a700 1 -- 172.21.15.84:0/2080842849 learned_addr learned my addr 172.21.15.84:0/2080842849 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb199b700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.332+0000 7f0fb219c700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac1f1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb119a700 1 -- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 msgr2=0x7f0fac1f1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:53.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb119a700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac1f1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb119a700 1 -- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 msgr2=0x7f0fac146a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb119a700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac146a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:53.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb119a700 1 -- 172.21.15.84:0/2080842849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8009550 con 0x7f0fac0b81c0 2023-07-06T21:42:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb119a700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 0x7f0fac15a930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009b60 tx=0x7f0f9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c004510 con 0x7f0fac0b81c0 2023-07-06T21:42:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c01f070 con 0x7f0fac0b81c0 2023-07-06T21:42:53.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c0094a0 con 0x7f0fac0b81c0 2023-07-06T21:42:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c0038e0 con 0x7f0fac0b81c0 2023-07-06T21:42:53.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.333+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac146f50 con 0x7f0fac0b81c0 2023-07-06T21:42:53.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.334+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac0c8940 con 0x7f0fac0b81c0 2023-07-06T21:42:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.335+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f0f9c003a40 con 0x7f0fac0b81c0 2023-07-06T21:42:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.336+0000 7f0fa27fc700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0f9006ace0 0x7f0f9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:53.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.336+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f0f9c021070 con 0x7f0fac0b81c0 2023-07-06T21:42:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.336+0000 7f0fb199b700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0f9006ace0 0x7f0f9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:53.343 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.337+0000 7f0fb199b700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0f9006ace0 0x7f0f9006d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8006d30 tx=0x7f0fa80092c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:53.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.341+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f0fac0c8940 con 0x7f0fac0b81c0 2023-07-06T21:42:53.584 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:53.582+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0fac0c8940 con 0x7f0fac0b81c0 2023-07-06T21:42:54.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.110+0000 7f0fa27fc700 1 -- 172.21.15.84:0/2080842849 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0fac0c8940 con 0x7f0fac0b81c0 2023-07-06T21:42:54.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0f9006ace0 msgr2=0x7f0f9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0f9006ace0 0x7f0f9006d190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8006d30 tx=0x7f0fa80092c0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 msgr2=0x7f0fac15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 0x7f0fac15a930 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c009b60 tx=0x7f0f9c004750 comp rx=0 tx=0).stop 2023-07-06T21:42:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 shutdown_connections 2023-07-06T21:42:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f0f9006ace0 0x7f0f9006d190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0fac0ceda0 0x7f0fac1f1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0fac0b1a80 0x7f0fac146a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 --2- 172.21.15.84:0/2080842849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0fac0b81c0 0x7f0fac15a930 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 >> 172.21.15.84:0/2080842849 conn(0x7f0fac06b2b0 msgr2=0x7f0fac0b3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 shutdown_connections 2023-07-06T21:42:54.117 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.113+0000 7f0fb3bff700 1 -- 172.21.15.84:0/2080842849 wait complete. 2023-07-06T21:42:54.135 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-07-06T21:42:54.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.333+0000 7f174a7ff700 1 Processor -- start 2023-07-06T21:42:54.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.333+0000 7f174a7ff700 1 -- start start 2023-07-06T21:42:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.334+0000 7f174a7ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1744160ac0 0x7f17440cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.334+0000 7f174a7ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f17440cfe20 0x7f174415fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.334+0000 7f174a7ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17440ceef0 con 0x7f17440d0ea0 2023-07-06T21:42:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.334+0000 7f174a7ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440cf450 con 0x7f1744160ac0 2023-07-06T21:42:54.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.334+0000 7f174a7ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440cf120 con 0x7f17440cfe20 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1744160ac0 0x7f17440cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f1744160ac0 0x7f17440cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44364/0 (socket says 172.21.15.84:44364) 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 -- 172.21.15.84:0/15682205 learned_addr learned my addr 172.21.15.84:0/15682205 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f1748d9c700 1 --2- 172.21.15.84:0/15682205 >> v2:172.21.15.107:3300/0 conn(0x7f17440cfe20 0x7f174415fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 -- 172.21.15.84:0/15682205 >> v2:172.21.15.107:3300/0 conn(0x7f17440cfe20 msgr2=0x7f174415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 --2- 172.21.15.84:0/15682205 >> v2:172.21.15.107:3300/0 conn(0x7f17440cfe20 0x7f174415fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 -- 172.21.15.84:0/15682205 >> v1:172.21.15.107:6789/0 conn(0x7f17440d0ea0 legacy=0x7f17440d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:54.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 -- 172.21.15.84:0/15682205 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440cd420 con 0x7f1744160ac0 2023-07-06T21:42:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.335+0000 7f17437fe700 1 --2- 172.21.15.84:0/15682205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744160ac0 0x7f17440cf8e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f17440c9d00 tx=0x7f1738009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28ca7e27695297e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f1742ffd700 1 -- 172.21.15.84:0/15682205 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800b870 con 0x7f1744160ac0 2023-07-06T21:42:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f1742ffd700 1 -- 172.21.15.84:0/15682205 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173800b9d0 con 0x7f1744160ac0 2023-07-06T21:42:54.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f1742ffd700 1 -- 172.21.15.84:0/15682205 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173800bd60 con 0x7f1744160ac0 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 -- 172.21.15.84:0/15682205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744160ac0 msgr2=0x7f17440cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 --2- 172.21.15.84:0/15682205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744160ac0 0x7f17440cf8e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f17440c9d00 tx=0x7f1738009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 -- 172.21.15.84:0/15682205 shutdown_connections 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 --2- 172.21.15.84:0/15682205 >> v2:172.21.15.107:3300/0 conn(0x7f17440cfe20 0x7f174415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 --2- 172.21.15.84:0/15682205 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1744160ac0 0x7f17440cf8e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 -- 172.21.15.84:0/15682205 >> 172.21.15.84:0/15682205 conn(0x7f17440715c0 msgr2=0x7f1744071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:54.341 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 -- 172.21.15.84:0/15682205 shutdown_connections 2023-07-06T21:42:54.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.336+0000 7f174a7ff700 1 -- 172.21.15.84:0/15682205 wait complete. 2023-07-06T21:42:54.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.338+0000 7f174a7ff700 1 Processor -- start 2023-07-06T21:42:54.342 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.338+0000 7f174a7ff700 1 -- start start 2023-07-06T21:42:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f174a7ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17440cfe20 0x7f17440cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f174a7ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1744160ac0 0x7f1744150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:54.344 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f174a7ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 0x7f174414e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f174a7ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1744150c80 con 0x7f174414cfb0 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f174a7ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174414dcf0 con 0x7f17440cfe20 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f174a7ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17440cc720 con 0x7f1744160ac0 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f1748d9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 0x7f174414e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f1748d9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 0x7f174414e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44378/0 (socket says 172.21.15.84:44378) 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f1748d9c700 1 -- 172.21.15.84:0/1634891465 learned_addr learned my addr 172.21.15.84:0/1634891465 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:54.345 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.339+0000 7f1743fff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17440cfe20 0x7f17440cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f17437fe700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1744160ac0 0x7f1744150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1748d9c700 1 -- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1744160ac0 msgr2=0x7f1744150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1748d9c700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1744160ac0 0x7f1744150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1748d9c700 1 -- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17440cfe20 msgr2=0x7f17440cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1748d9c700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17440cfe20 0x7f17440cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1748d9c700 1 -- 172.21.15.84:0/1634891465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738009550 con 0x7f174414cfb0 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1748d9c700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 0x7f174414e180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f173c007e30 tx=0x7f173c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:54.346 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c00ed30 con 0x7f174414cfb0 2023-07-06T21:42:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f173c00ee90 con 0x7f174414cfb0 2023-07-06T21:42:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173c014660 con 0x7f174414cfb0 2023-07-06T21:42:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.340+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17440cca00 con 0x7f174414cfb0 2023-07-06T21:42:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.341+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17440ccfc0 con 0x7f174414cfb0 2023-07-06T21:42:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.341+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174414de80 con 0x7f174414cfb0 2023-07-06T21:42:54.347 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.342+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f173c0147c0 con 0x7f174414cfb0 2023-07-06T21:42:54.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.343+0000 7f1740ff9700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f173006ace0 0x7f173006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:54.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.343+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6148+0+0 (secure 0 0 0) 0x7f173c089c30 con 0x7f174414cfb0 2023-07-06T21:42:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.343+0000 7f1743fff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f173006ace0 0x7f173006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:54.349 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.344+0000 7f1743fff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f173006ace0 0x7f173006d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1734000fb0 tx=0x7f1734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:54.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.348+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f173c058670 con 0x7f174414cfb0 2023-07-06T21:42:54.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:54.545+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f174404e1d0 con 0x7f174414cfb0 2023-07-06T21:42:55.126 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.124+0000 7f1740ff9700 1 -- 172.21.15.84:0/1634891465 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7f173c058200 con 0x7f174414cfb0 2023-07-06T21:42:55.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.127+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f173006ace0 msgr2=0x7f173006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.127+0000 7f174a7ff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f173006ace0 0x7f173006d190 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1734000fb0 tx=0x7f1734009040 comp rx=0 tx=0).stop 2023-07-06T21:42:55.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.127+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 msgr2=0x7f174414e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.127+0000 7f174a7ff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 0x7f174414e180 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f173c007e30 tx=0x7f173c00e3c0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 shutdown_connections 2023-07-06T21:42:55.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f173006ace0 0x7f173006d190 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.130 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f174414cfb0 0x7f174414e180 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f17440cfe20 0x7f17440cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 --2- 172.21.15.84:0/1634891465 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1744160ac0 0x7f1744150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 >> 172.21.15.84:0/1634891465 conn(0x7f17440715c0 msgr2=0x7f174415c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:55.131 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 shutdown_connections 2023-07-06T21:42:55.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.128+0000 7f174a7ff700 1 -- 172.21.15.84:0/1634891465 wait complete. 2023-07-06T21:42:55.132 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2023-07-06T21:42:55.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-07-06T21:42:55.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.346+0000 7f5da3fff700 1 Processor -- start 2023-07-06T21:42:55.348 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.346+0000 7f5da3fff700 1 -- start start 2023-07-06T21:42:55.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.347+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 0x7f5d9c0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.347+0000 7f5da3fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0ceef0 con 0x7f5d9c0d0ea0 2023-07-06T21:42:55.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.347+0000 7f5da3fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0cf450 con 0x7f5d9c0cfe20 2023-07-06T21:42:55.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.347+0000 7f5da3fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cf120 con 0x7f5d9c160ac0 2023-07-06T21:42:55.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.347+0000 7f5da1d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5d9c0d0ea0 0x7f5d9c0d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59978/0 (socket says 172.21.15.84:59978) 2023-07-06T21:42:55.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.347+0000 7f5da1d9b700 1 -- 172.21.15.84:0/2940446864 learned_addr learned my addr 172.21.15.84:0/2940446864 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:55.354 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da159a700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 0x7f5d9c0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586746452 0 0) 0x7f5d9c0ceef0 con 0x7f5d9c0d0ea0 2023-07-06T21:42:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d880035b0 con 0x7f5d9c0d0ea0 2023-07-06T21:42:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743146938 0 0) 0x7f5d9c0cf450 con 0x7f5d9c0cfe20 2023-07-06T21:42:55.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d9c0ceef0 con 0x7f5d9c0cfe20 2023-07-06T21:42:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 447868952 0 0) 0x7f5d880035b0 con 0x7f5d9c0d0ea0 2023-07-06T21:42:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 msgr2=0x7f5d9c0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 0x7f5d9c0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 >> v1:172.21.15.107:6789/0 conn(0x7f5d9c0cfe20 legacy=0x7f5d9c15fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0cd420 con 0x7f5d9c0d0ea0 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f5d980017f0 con 0x7f5d9c0d0ea0 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 >> v1:172.21.15.84:6789/0 conn(0x7f5d9c0d0ea0 legacy=0x7f5d9c0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d88003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 0x7f5d88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d88004d90 0x7f5d88005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cf120 con 0x7f5d88003bc0 2023-07-06T21:42:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8800bc20 con 0x7f5d88004d90 2023-07-06T21:42:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8800bde0 con 0x7f5d9c160ac0 2023-07-06T21:42:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d88003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da159a700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 0x7f5d88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.348+0000 7f5da259c700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d88004d90 0x7f5d88005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da1d9b700 1 -- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 msgr2=0x7f5d88004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c160ac0 0x7f5d88004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da1d9b700 1 -- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d88004d90 msgr2=0x7f5d88005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d88004d90 0x7f5d88005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da1d9b700 1 -- 172.21.15.84:0/2940446864 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d880035b0 con 0x7f5d88003bc0 2023-07-06T21:42:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d88003fb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c110 tx=0x7f5d88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62590ad5fe86946 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:55.359 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9800ccb0 con 0x7f5d88003bc0 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9800ce10 con 0x7f5d88003bc0 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.349+0000 7f5da0d99700 1 -- 172.21.15.84:0/2940446864 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98007310 con 0x7f5d88003bc0 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da3fff700 1 -- 172.21.15.84:0/2940446864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 msgr2=0x7f5d88003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da3fff700 1 --2- 172.21.15.84:0/2940446864 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d88003fb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800c110 tx=0x7f5d88005880 comp rx=0 tx=0).stop 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da259c700 1 -- 172.21.15.84:0/2940446864 reap_dead start 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da3fff700 1 -- 172.21.15.84:0/2940446864 shutdown_connections 2023-07-06T21:42:55.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da3fff700 1 -- 172.21.15.84:0/2940446864 >> 172.21.15.84:0/2940446864 conn(0x7f5d9c0715c0 msgr2=0x7f5d9c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da3fff700 1 -- 172.21.15.84:0/2940446864 shutdown_connections 2023-07-06T21:42:55.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.350+0000 7f5da3fff700 1 -- 172.21.15.84:0/2940446864 wait complete. 2023-07-06T21:42:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.352+0000 7f5da3fff700 1 Processor -- start 2023-07-06T21:42:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.352+0000 7f5da3fff700 1 -- start start 2023-07-06T21:42:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d9c0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c0c5b20 0x7f5d9c0b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c0b7160 0x7f5d9c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da3fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0b7600 con 0x7f5d88003bc0 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da3fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0c5f90 con 0x7f5d9c0b7160 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da3fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0cbe90 con 0x7f5d9c0c5b20 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d9c0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d9c0cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44384/0 (socket says 172.21.15.84:44384) 2023-07-06T21:42:55.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 -- 172.21.15.84:0/102378627 learned_addr learned my addr 172.21.15.84:0/102378627 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da159a700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c0c5b20 0x7f5d9c0b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da259c700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c0b7160 0x7f5d9c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 -- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c0c5b20 msgr2=0x7f5d9c0b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c0c5b20 0x7f5d9c0b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 -- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c0b7160 msgr2=0x7f5d9c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c0b7160 0x7f5d9c0ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:55.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.353+0000 7f5da1d9b700 1 -- 172.21.15.84:0/102378627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0cb580 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.354+0000 7f5da1d9b700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d9c0cc320 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d98001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.354+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d980074e0 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.354+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98007640 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.354+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0cb840 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.354+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98005a70 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.354+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c0ca600 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.355+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c04e1d0 con 0x7f5d88003bc0 2023-07-06T21:42:55.366 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.356+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5d98019070 con 0x7f5d88003bc0 2023-07-06T21:42:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.357+0000 7f5d927fc700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5d8006ace0 0x7f5d8006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.357+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f5d9808a210 con 0x7f5d88003bc0 2023-07-06T21:42:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.357+0000 7f5da159a700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5d8006ace0 0x7f5d8006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.358+0000 7f5da159a700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5d8006ace0 0x7f5d8006d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0c63e0 tx=0x7f5d8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:55.367 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.361+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5d98055410 con 0x7f5d88003bc0 2023-07-06T21:42:55.558 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:55.556+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5d9c15e850 con 0x7f5d88003bc0 2023-07-06T21:42:56.129 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.127+0000 7f5d927fc700 1 -- 172.21.15.84:0/102378627 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5d98058a30 con 0x7f5d88003bc0 2023-07-06T21:42:56.133 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5d8006ace0 msgr2=0x7f5d8006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5d8006ace0 0x7f5d8006d190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0c63e0 tx=0x7f5d8c008040 comp rx=0 tx=0).stop 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 msgr2=0x7f5d9c0cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d9c0cc320 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f5d98000c00 tx=0x7f5d98001ab0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 shutdown_connections 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5d8006ace0 0x7f5d8006d190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d9c0b7160 0x7f5d9c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5d88003bc0 0x7f5d9c0cc320 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 --2- 172.21.15.84:0/102378627 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d9c0c5b20 0x7f5d9c0b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.131+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 >> 172.21.15.84:0/102378627 conn(0x7f5d9c0715c0 msgr2=0x7f5d9c15ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:56.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.132+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 shutdown_connections 2023-07-06T21:42:56.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.132+0000 7f5da3fff700 1 -- 172.21.15.84:0/102378627 wait complete. 2023-07-06T21:42:56.136 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2023-07-06T21:42:56.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-07-06T21:42:56.350 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.348+0000 7ffb357ff700 1 Processor -- start 2023-07-06T21:42:56.351 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.349+0000 7ffb357ff700 1 -- start start 2023-07-06T21:42:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb357ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffb300b81c0 0x7ffb300cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb357ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb357ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb300d1830 con 0x7ffb300ceda0 2023-07-06T21:42:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb357ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb300cf330 con 0x7ffb300b81c0 2023-07-06T21:42:56.352 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb357ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300d1270 con 0x7ffb300b1a80 2023-07-06T21:42:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb2f7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ffb300ceda0 0x7ffb3015ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:59988/0 (socket says 172.21.15.84:59988) 2023-07-06T21:42:56.353 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb2f7fe700 1 -- 172.21.15.84:0/1555991150 learned_addr learned my addr 172.21.15.84:0/1555991150 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:56.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb2e7fc700 1 --2- 172.21.15.84:0/1555991150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.355 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb2effd700 1 --2- 172.21.15.84:0/1555991150 >> v2:172.21.15.107:3300/0 conn(0x7ffb300b81c0 0x7ffb300cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb2dffb700 1 -- 172.21.15.84:0/1555991150 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975109217 0 0) 0x7ffb300d1830 con 0x7ffb300ceda0 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.350+0000 7ffb2dffb700 1 -- 172.21.15.84:0/1555991150 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb180035b0 con 0x7ffb300ceda0 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1555991150 >> v2:172.21.15.107:3300/0 conn(0x7ffb300b81c0 msgr2=0x7ffb300cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2e7fc700 1 --2- 172.21.15.84:0/1555991150 >> v2:172.21.15.107:3300/0 conn(0x7ffb300b81c0 0x7ffb300cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1555991150 >> v1:172.21.15.84:6789/0 conn(0x7ffb300ceda0 legacy=0x7ffb3015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1555991150 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb300d1550 con 0x7ffb300b1a80 2023-07-06T21:42:56.356 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2effd700 1 --2- 172.21.15.84:0/1555991150 >> v2:172.21.15.107:3300/0 conn(0x7ffb300b81c0 0x7ffb300cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2e7fc700 1 --2- 172.21.15.84:0/1555991150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300cfb40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009840 tx=0x7ffb24009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc73727d2cf87156 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2dffb700 1 -- 172.21.15.84:0/1555991150 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24011830 con 0x7ffb300b1a80 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2dffb700 1 -- 172.21.15.84:0/1555991150 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb24011990 con 0x7ffb300b1a80 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.351+0000 7ffb2dffb700 1 -- 172.21.15.84:0/1555991150 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24011d20 con 0x7ffb300b1a80 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 -- 172.21.15.84:0/1555991150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 msgr2=0x7ffb300cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1555991150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300cfb40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009840 tx=0x7ffb24009b50 comp rx=0 tx=0).stop 2023-07-06T21:42:56.357 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 -- 172.21.15.84:0/1555991150 shutdown_connections 2023-07-06T21:42:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1555991150 >> v2:172.21.15.107:3300/0 conn(0x7ffb300b81c0 0x7ffb300cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1555991150 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300cfb40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 -- 172.21.15.84:0/1555991150 >> 172.21.15.84:0/1555991150 conn(0x7ffb3006b2b0 msgr2=0x7ffb3006b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 -- 172.21.15.84:0/1555991150 shutdown_connections 2023-07-06T21:42:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.352+0000 7ffb357ff700 1 -- 172.21.15.84:0/1555991150 wait complete. 2023-07-06T21:42:56.358 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.354+0000 7ffb357ff700 1 Processor -- start 2023-07-06T21:42:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.354+0000 7ffb357ff700 1 -- start start 2023-07-06T21:42:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb357ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb357ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 0x7ffb301e6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.360 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb357ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb300ceda0 0x7ffb300c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb357ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb300c7d50 con 0x7ffb300b81c0 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb357ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301e53a0 con 0x7ffb300ceda0 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb357ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb301e5680 con 0x7ffb300b1a80 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 0x7ffb301e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 0x7ffb301e6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44396/0 (socket says 172.21.15.84:44396) 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1491939236 learned_addr learned my addr 172.21.15.84:0/1491939236 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:56.361 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2effd700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2f7fe700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb300ceda0 0x7ffb300c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 msgr2=0x7ffb300c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300c7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb300ceda0 msgr2=0x7ffb300c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb300ceda0 0x7ffb300c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2e7fc700 1 -- 172.21.15.84:0/1491939236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200094a0 con 0x7ffb300b81c0 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.355+0000 7ffb2f7fe700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb300ceda0 0x7ffb300c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:42:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.356+0000 7ffb2e7fc700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 0x7ffb301e6440 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ffb240090b0 tx=0x7ffb24004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.356+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb240036a0 con 0x7ffb300b81c0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.356+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2401f070 con 0x7ffb300b81c0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.356+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb24003a60 con 0x7ffb300b81c0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.356+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb240094a0 con 0x7ffb300b81c0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.356+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb301e5c40 con 0x7ffb300b81c0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.357+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3004e1d0 con 0x7ffb300b81c0 2023-07-06T21:42:56.363 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.358+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ffb24003bc0 con 0x7ffb300b81c0 2023-07-06T21:42:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.359+0000 7ffb177fe700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb1c06ad10 0x7ffb1c06d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.360+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ffb24021070 con 0x7ffb300b81c0 2023-07-06T21:42:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.360+0000 7ffb2effd700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb1c06ad10 0x7ffb1c06d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.364 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.360+0000 7ffb2effd700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb1c06ad10 0x7ffb1c06d1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009d70 tx=0x7ffb20009210 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:56.365 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.364+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ffb24055cc0 con 0x7ffb300b81c0 2023-07-06T21:42:56.562 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.560+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ffb300b4f30 con 0x7ffb300b81c0 2023-07-06T21:42:56.568 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.566+0000 7ffb177fe700 1 -- 172.21.15.84:0/1491939236 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7ffb240592e0 con 0x7ffb300b81c0 2023-07-06T21:42:56.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.568+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb1c06ad10 msgr2=0x7ffb1c06d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.568+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb1c06ad10 0x7ffb1c06d1c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009d70 tx=0x7ffb20009210 comp rx=0 tx=0).stop 2023-07-06T21:42:56.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.568+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 msgr2=0x7ffb301e6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.568+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 0x7ffb301e6440 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ffb240090b0 tx=0x7ffb24004750 comp rx=0 tx=0).stop 2023-07-06T21:42:56.571 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 shutdown_connections 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ffb1c06ad10 0x7ffb1c06d1c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb300ceda0 0x7ffb300c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb300b1a80 0x7ffb300c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 --2- 172.21.15.84:0/1491939236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffb300b81c0 0x7ffb301e6440 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 >> 172.21.15.84:0/1491939236 conn(0x7ffb3006b2b0 msgr2=0x7ffb3015ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 shutdown_connections 2023-07-06T21:42:56.572 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.569+0000 7ffb357ff700 1 -- 172.21.15.84:0/1491939236 wait complete. 2023-07-06T21:42:56.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2023-07-06T21:42:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.784+0000 7f626afff700 1 Processor -- start 2023-07-06T21:42:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.784+0000 7f626afff700 1 -- start start 2023-07-06T21:42:56.787 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.785+0000 7f626afff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f62640b83c0 0x7f62640cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.785+0000 7f626afff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f62640cfc60 0x7f62640d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.785+0000 7f626afff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640d0a80 con 0x7f62640d3710 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.785+0000 7f626afff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640b8790 con 0x7f62640b83c0 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.785+0000 7f626afff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640b4e70 con 0x7f62640cfc60 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6268d9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f62640b83c0 0x7f62640cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6263fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f62640d3710 0x7f62640cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60004/0 (socket says 172.21.15.84:60004) 2023-07-06T21:42:56.788 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6263fff700 1 -- 172.21.15.84:0/3268929798 learned_addr learned my addr 172.21.15.84:0/3268929798 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f626959c700 1 --2- 172.21.15.84:0/3268929798 >> v2:172.21.15.107:3300/0 conn(0x7f62640cfc60 0x7f62640d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6268d9b700 1 -- 172.21.15.84:0/3268929798 >> v2:172.21.15.107:3300/0 conn(0x7f62640cfc60 msgr2=0x7f62640d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6268d9b700 1 --2- 172.21.15.84:0/3268929798 >> v2:172.21.15.107:3300/0 conn(0x7f62640cfc60 0x7f62640d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6268d9b700 1 -- 172.21.15.84:0/3268929798 >> v1:172.21.15.84:6789/0 conn(0x7f62640d3710 legacy=0x7f62640cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6268d9b700 1 -- 172.21.15.84:0/3268929798 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640b5150 con 0x7f62640b83c0 2023-07-06T21:42:56.790 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f62637fe700 1 -- 172.21.15.84:0/3268929798 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751044102 0 0) 0x7f62640d0a80 con 0x7f62640d3710 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f6268d9b700 1 --2- 172.21.15.84:0/3268929798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640b83c0 0x7f62640cd200 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f62640d0030 tx=0x7f6258009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb03dfb18a9a516c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.786+0000 7f62637fe700 1 -- 172.21.15.84:0/3268929798 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258011830 con 0x7f62640b83c0 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f62637fe700 1 -- 172.21.15.84:0/3268929798 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6258011990 con 0x7f62640b83c0 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f62637fe700 1 -- 172.21.15.84:0/3268929798 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6258011d20 con 0x7f62640b83c0 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 -- 172.21.15.84:0/3268929798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640b83c0 msgr2=0x7f62640cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 --2- 172.21.15.84:0/3268929798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640b83c0 0x7f62640cd200 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f62640d0030 tx=0x7f6258009b20 comp rx=0 tx=0).stop 2023-07-06T21:42:56.791 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 -- 172.21.15.84:0/3268929798 shutdown_connections 2023-07-06T21:42:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 --2- 172.21.15.84:0/3268929798 >> v2:172.21.15.107:3300/0 conn(0x7f62640cfc60 0x7f62640d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 --2- 172.21.15.84:0/3268929798 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640b83c0 0x7f62640cd200 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 -- 172.21.15.84:0/3268929798 >> 172.21.15.84:0/3268929798 conn(0x7f626406b350 msgr2=0x7f626406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 -- 172.21.15.84:0/3268929798 shutdown_connections 2023-07-06T21:42:56.792 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.787+0000 7f626afff700 1 -- 172.21.15.84:0/3268929798 wait complete. 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.789+0000 7f626afff700 1 Processor -- start 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.789+0000 7f626afff700 1 -- start start 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626afff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62640b83c0 0x7f6264153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626afff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62640cfc60 0x7f626415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626afff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 0x7f62640b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626afff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415ca30 con 0x7f62640d3710 2023-07-06T21:42:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626afff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415cd10 con 0x7f62640cfc60 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626afff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626415cff0 con 0x7f62640b83c0 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626959c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 0x7f62640b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626959c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 0x7f62640b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44412/0 (socket says 172.21.15.84:44412) 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f626959c700 1 -- 172.21.15.84:0/442150367 learned_addr learned my addr 172.21.15.84:0/442150367 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f6263fff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62640cfc60 0x7f626415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.790+0000 7f6268d9b700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62640b83c0 0x7f6264153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.795 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626959c700 1 -- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62640b83c0 msgr2=0x7f6264153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626959c700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62640b83c0 0x7f6264153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626959c700 1 -- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62640cfc60 msgr2=0x7f626415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626959c700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62640cfc60 0x7f626415dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626959c700 1 -- 172.21.15.84:0/442150367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62580094a0 con 0x7f62640d3710 2023-07-06T21:42:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f6268d9b700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62640b83c0 0x7f6264153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:56.796 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626959c700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 0x7f62640b5970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f625c007d40 tx=0x7f625c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c00ede0 con 0x7f62640d3710 2023-07-06T21:42:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c014420 con 0x7f62640d3710 2023-07-06T21:42:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626415d2d0 con 0x7f62640d3710 2023-07-06T21:42:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0147d0 con 0x7f62640d3710 2023-07-06T21:42:56.797 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.791+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626415e010 con 0x7f62640d3710 2023-07-06T21:42:56.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.792+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626415d7f0 con 0x7f62640d3710 2023-07-06T21:42:56.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.793+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f625c014930 con 0x7f62640d3710 2023-07-06T21:42:56.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.794+0000 7f62617fa700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f624806ace0 0x7f624806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:56.798 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.795+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f625c089930 con 0x7f62640d3710 2023-07-06T21:42:56.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.795+0000 7f6268d9b700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f624806ace0 0x7f624806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:56.799 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.795+0000 7f6268d9b700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f624806ace0 0x7f624806d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6258009b60 tx=0x7f6258008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:56.800 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:56.798+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f626415d7f0 con 0x7f62640d3710 2023-07-06T21:42:57.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.077+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) v1 -- 0x7f626415d7f0 con 0x7f62640d3710 2023-07-06T21:42:57.132 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.130+0000 7f62617fa700 1 -- 172.21.15.84:0/442150367 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v214) v1 ==== 134+0+0 (secure 0 0 0) 0x7f626415d7f0 con 0x7f62640d3710 2023-07-06T21:42:57.134 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f624806ace0 msgr2=0x7f624806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:57.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f624806ace0 0x7f624806d190 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6258009b60 tx=0x7f6258008030 comp rx=0 tx=0).stop 2023-07-06T21:42:57.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 msgr2=0x7f62640b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:57.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 0x7f62640b5970 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f625c007d40 tx=0x7f625c00e3c0 comp rx=0 tx=0).stop 2023-07-06T21:42:57.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 shutdown_connections 2023-07-06T21:42:57.135 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f624806ace0 0x7f624806d190 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:57.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62640d3710 0x7f62640b5970 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:57.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62640b83c0 0x7f6264153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:57.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 --2- 172.21.15.84:0/442150367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62640cfc60 0x7f626415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:57.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 >> 172.21.15.84:0/442150367 conn(0x7f626406b350 msgr2=0x7f62640b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:57.136 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 shutdown_connections 2023-07-06T21:42:57.137 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:57.132+0000 7f626afff700 1 -- 172.21.15.84:0/442150367 wait complete. 2023-07-06T21:42:57.153 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2023-07-06T21:42:57.153 INFO:tasks.cephfs_test_runner: 2023-07-06T21:42:57.153 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T21:42:57.154 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-06T21:42:57.154 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T21:42:57.154 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T21:42:57.154 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-07-06T21:42:57.154 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-07-06T21:42:57.155 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks/cephfs/mount.py", line 771, in run_python 2023-07-06T21:42:57.155 INFO:tasks.cephfs_test_runner: p.wait() 2023-07-06T21:42:57.155 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait 2023-07-06T21:42:57.155 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-06T21:42:57.155 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-06T21:42:57.156 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-06T21:42:57.156 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi084 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-07-06T21:42:57.156 INFO:tasks.cephfs_test_runner: 2023-07-06T21:42:57.156 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T21:42:57.157 INFO:tasks.cephfs_test_runner:Ran 2 tests in 168.225s 2023-07-06T21:42:57.157 INFO:tasks.cephfs_test_runner: 2023-07-06T21:42:57.157 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-07-06T21:42:57.157 INFO:tasks.cephfs_test_runner: 2023-07-06T21:42:57.157 INFO:tasks.cephfs_test_runner:====================================================================== 2023-07-06T21:42:57.158 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-06T21:42:57.158 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-07-06T21:42:57.158 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-07-06T21:42:57.158 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2023-07-06T21:42:57.158 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2023-07-06T21:42:57.159 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks/cephfs/mount.py", line 771, in run_python 2023-07-06T21:42:57.159 INFO:tasks.cephfs_test_runner: p.wait() 2023-07-06T21:42:57.159 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 161, in wait 2023-07-06T21:42:57.159 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-07-06T21:42:57.159 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-07-06T21:42:57.160 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-07-06T21:42:57.160 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi084 with status 1: 'adjust-ulimits daemon-helper kill python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2023-07-06T21:42:57.160 INFO:tasks.cephfs_test_runner: 2023-07-06T21:42:57.160 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-06T21:42:57.243 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=02002e561b714793af73106e6a1af032 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b53f236c1077f1d295baafb9e0ca9a4b447019d7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_0a781ef0807ae9473fb1306ef07036226ca4f1a6/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2023-07-06T21:42:57.246 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-07-06T21:42:57.257 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-07-06T21:42:57.338 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-07-06T21:42:57.339 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:57.339 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:57.384 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:42:57.385 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2023-07-06T21:42:57.385 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:57.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345720k,nr_inodes=4086430,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275008k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-07-06T21:42:57.468 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:42:57.468 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:57.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:42:57.516 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2023-07-06T21:42:57.516 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-06T21:42:57.600 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345556k,nr_inodes=4086389,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274976k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-07-06T21:42:57.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:57.601 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2023-07-06T21:42:57.644 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-07-06T21:42:57.644 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-07-06T21:42:57.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:57.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-07-06T21:42:57.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:57.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-07-06T21:42:57.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:57.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2023-07-06T21:42:57.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:57.942 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2023-07-06T21:42:57.990 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-07-06T21:42:57.990 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-07-06T21:42:57.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:57.992 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-07-06T21:42:58.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:58.063 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-07-06T21:42:58.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-06T21:42:58.148 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2023-07-06T21:42:58.251 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-07-06T21:42:58.264 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-07-06T21:42:58.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:42:58.460 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.458+0000 7f406fdff700 1 Processor -- start 2023-07-06T21:42:58.461 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.458+0000 7f406fdff700 1 -- start start 2023-07-06T21:42:58.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406fdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4068070230 0x7f40680706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406fdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40680c9c10 0x7f40680ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406fdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406815d930 con 0x7f4068159130 2023-07-06T21:42:58.463 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406fdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4068070bf0 con 0x7f40680c9c10 2023-07-06T21:42:58.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406fdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406815cd20 con 0x7f4068070230 2023-07-06T21:42:58.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40680c9c10 0x7f40680ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f40680c9c10 0x7f40680ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44428/0 (socket says 172.21.15.84:44428) 2023-07-06T21:42:58.464 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 -- 172.21.15.84:0/2886903460 learned_addr learned my addr 172.21.15.84:0/2886903460 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406d39a700 1 --2- 172.21.15.84:0/2886903460 >> v2:172.21.15.107:3300/0 conn(0x7f4068070230 0x7f40680706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 -- 172.21.15.84:0/2886903460 >> v2:172.21.15.107:3300/0 conn(0x7f4068070230 msgr2=0x7f40680706b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 --2- 172.21.15.84:0/2886903460 >> v2:172.21.15.107:3300/0 conn(0x7f4068070230 0x7f40680706b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 -- 172.21.15.84:0/2886903460 >> v1:172.21.15.107:6789/0 conn(0x7f4068159130 legacy=0x7f40680c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.465 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406e39c700 1 -- 172.21.15.84:0/2886903460 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406815d000 con 0x7f40680c9c10 2023-07-06T21:42:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.460+0000 7f406cb99700 1 -- 172.21.15.84:0/2886903460 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159905818 0 0) 0x7f406815d930 con 0x7f4068159130 2023-07-06T21:42:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.461+0000 7f406e39c700 1 --2- 172.21.15.84:0/2886903460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40680c9c10 0x7f40680ca010 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40680713d0 tx=0x7f406000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40d7e6cd39ee0c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.461+0000 7f406cb99700 1 -- 172.21.15.84:0/2886903460 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000e070 con 0x7f40680c9c10 2023-07-06T21:42:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.461+0000 7f406cb99700 1 -- 172.21.15.84:0/2886903460 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406000b910 con 0x7f40680c9c10 2023-07-06T21:42:58.467 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.461+0000 7f406cb99700 1 -- 172.21.15.84:0/2886903460 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406000bcc0 con 0x7f40680c9c10 2023-07-06T21:42:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.461+0000 7f406fdff700 1 -- 172.21.15.84:0/2886903460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40680c9c10 msgr2=0x7f40680ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.461+0000 7f406fdff700 1 --2- 172.21.15.84:0/2886903460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40680c9c10 0x7f40680ca010 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f40680713d0 tx=0x7f406000b330 comp rx=0 tx=0).stop 2023-07-06T21:42:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.462+0000 7f406fdff700 1 -- 172.21.15.84:0/2886903460 shutdown_connections 2023-07-06T21:42:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.462+0000 7f406fdff700 1 --2- 172.21.15.84:0/2886903460 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40680c9c10 0x7f40680ca010 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.462+0000 7f406fdff700 1 --2- 172.21.15.84:0/2886903460 >> v2:172.21.15.107:3300/0 conn(0x7f4068070230 0x7f40680706b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.462+0000 7f406fdff700 1 -- 172.21.15.84:0/2886903460 >> 172.21.15.84:0/2886903460 conn(0x7f40680cf3d0 msgr2=0x7f40680cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.462+0000 7f406fdff700 1 -- 172.21.15.84:0/2886903460 shutdown_connections 2023-07-06T21:42:58.470 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.462+0000 7f406fdff700 1 -- 172.21.15.84:0/2886903460 wait complete. 2023-07-06T21:42:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.464+0000 7f406fdff700 1 Processor -- start 2023-07-06T21:42:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.464+0000 7f406fdff700 1 -- start start 2023-07-06T21:42:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406fdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4068070230 0x7f406814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.471 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406fdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40680c9c10 0x7f40680c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406fdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 0x7f40680af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406fdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40680b2800 con 0x7f4068159130 2023-07-06T21:42:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406fdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406814d660 con 0x7f40680c9c10 2023-07-06T21:42:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406fdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40680b1c10 con 0x7f4068070230 2023-07-06T21:42:58.472 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406e39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 0x7f40680af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406db9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4068070230 0x7f406814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406e39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 0x7f40680af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44434/0 (socket says 172.21.15.84:44434) 2023-07-06T21:42:58.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406e39c700 1 -- 172.21.15.84:0/3806286464 learned_addr learned my addr 172.21.15.84:0/3806286464 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:58.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.465+0000 7f406d39a700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40680c9c10 0x7f40680c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.473 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406e39c700 1 -- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4068070230 msgr2=0x7f406814d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406e39c700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4068070230 0x7f406814d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406e39c700 1 -- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40680c9c10 msgr2=0x7f40680c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406e39c700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40680c9c10 0x7f40680c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.474 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406e39c700 1 -- 172.21.15.84:0/3806286464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4060009b30 con 0x7f4068159130 2023-07-06T21:42:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406e39c700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 0x7f40680af640 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f406806b750 tx=0x7f4060004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4060004880 con 0x7f4068159130 2023-07-06T21:42:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40600099b0 con 0x7f4068159130 2023-07-06T21:42:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406001c680 con 0x7f4068159130 2023-07-06T21:42:58.475 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40680b1ed0 con 0x7f4068159130 2023-07-06T21:42:58.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.466+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40680b2490 con 0x7f4068159130 2023-07-06T21:42:58.476 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.467+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40680ad820 con 0x7f4068159130 2023-07-06T21:42:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.468+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f406001c7e0 con 0x7f4068159130 2023-07-06T21:42:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.469+0000 7f405e7fc700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f404c06ace0 0x7f404c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.477 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.469+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f406008ace0 con 0x7f4068159130 2023-07-06T21:42:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.469+0000 7f406db9b700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f404c06ace0 0x7f404c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.470+0000 7f406db9b700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f404c06ace0 0x7f404c06d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4064007fd0 tx=0x7f4064009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.473+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f4060056120 con 0x7f4068159130 2023-07-06T21:42:58.668 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.666+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f406815d5c0 con 0x7f404c06ace0 2023-07-06T21:42:58.671 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.669+0000 7f405e7fc700 1 -- 172.21.15.84:0/3806286464 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72642 (secure 0 0 0) 0x7f40640015d0 con 0x7f404c06ace0 2023-07-06T21:42:58.672 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f404c06ace0 msgr2=0x7f404c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f404c06ace0 0x7f404c06d190 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4064007fd0 tx=0x7f4064009040 comp rx=0 tx=0).stop 2023-07-06T21:42:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 msgr2=0x7f40680af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 0x7f40680af640 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f406806b750 tx=0x7f4060004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 shutdown_connections 2023-07-06T21:42:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f404c06ace0 0x7f404c06d190 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4068159130 0x7f40680af640 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4068070230 0x7f406814d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 --2- 172.21.15.84:0/3806286464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40680c9c10 0x7f40680c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 >> 172.21.15.84:0/3806286464 conn(0x7f40680cf3d0 msgr2=0x7f4068069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:58.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 shutdown_connections 2023-07-06T21:42:58.677 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.672+0000 7f406fdff700 1 -- 172.21.15.84:0/3806286464 wait complete. 2023-07-06T21:42:58.678 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:42:58.692 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2023-07-06T21:42:58.179873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221972,"kb_used_data":6356,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754752748,"statfs":{"total":773094113280,"available":772866813952,"internally_reserved":0,"allocated":6508544,"data_stored":3510599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14363,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000966"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.578034+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:42:56.578034+0000","last_peered":"2023-07-06T21:42:56.578034+0000","last_clean":"2023-07-06T21:42:56.578034+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:42:56.578034+0000","last_undegraded":"2023-07-06T21:42:56.578034+0000","last_fullsized":"2023-07-06T21:42:56.578034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:02:13.288211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128555+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:42:56.128555+0000","last_peered":"2023-07-06T21:42:56.128555+0000","last_clean":"2023-07-06T21:42:56.128555+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:42:56.128555+0000","last_undegraded":"2023-07-06T21:42:56.128555+0000","last_fullsized":"2023-07-06T21:42:56.128555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:57.209997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.837111+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:42:56.837111+0000","last_peered":"2023-07-06T21:42:56.837111+0000","last_clean":"2023-07-06T21:42:56.837111+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:42:56.837111+0000","last_undegraded":"2023-07-06T21:42:56.837111+0000","last_fullsized":"2023-07-06T21:42:56.837111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:27:05.384819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.130367+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:42:56.130367+0000","last_peered":"2023-07-06T21:42:56.130367+0000","last_clean":"2023-07-06T21:42:56.130367+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:42:56.130367+0000","last_undegraded":"2023-07-06T21:42:56.130367+0000","last_fullsized":"2023-07-06T21:42:56.130367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:23:36.326578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141735+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:42:57.141735+0000","last_peered":"2023-07-06T21:42:57.141735+0000","last_clean":"2023-07-06T21:42:57.141735+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:42:57.141735+0000","last_undegraded":"2023-07-06T21:42:57.141735+0000","last_fullsized":"2023-07-06T21:42:57.141735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:20:39.183176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141546+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:42:57.141546+0000","last_peered":"2023-07-06T21:42:57.141546+0000","last_clean":"2023-07-06T21:42:57.141546+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:42:57.141546+0000","last_undegraded":"2023-07-06T21:42:57.141546+0000","last_fullsized":"2023-07-06T21:42:57.141546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:01:48.302621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.140311+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:42:57.140311+0000","last_peered":"2023-07-06T21:42:57.140311+0000","last_clean":"2023-07-06T21:42:57.140311+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:42:57.140311+0000","last_undegraded":"2023-07-06T21:42:57.140311+0000","last_fullsized":"2023-07-06T21:42:57.140311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:28.126913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.576736+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:42:56.576736+0000","last_peered":"2023-07-06T21:42:56.576736+0000","last_clean":"2023-07-06T21:42:56.576736+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:42:56.576736+0000","last_undegraded":"2023-07-06T21:42:56.576736+0000","last_fullsized":"2023-07-06T21:42:56.576736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:02:07.005730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128278+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:42:56.128278+0000","last_peered":"2023-07-06T21:42:56.128278+0000","last_clean":"2023-07-06T21:42:56.128278+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:42:56.128278+0000","last_undegraded":"2023-07-06T21:42:56.128278+0000","last_fullsized":"2023-07-06T21:42:56.128278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:44.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1253376,"data_stored":879509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.53900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.101,"5min":1.28,"15min":1.28},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.995,"15min":0.995},"last":0.751}]},{"osd":1,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.80600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89500000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69699999999999995,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.872,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.972,"15min":1.972},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.379,"15min":1.379},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79500000000000004,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85199999999999998,"5min":3.569,"15min":3.569},"last":0.872},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.76200000000000001,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.75800000000000001,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99199999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.80000000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82999999999999996,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88500000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.85299999999999998},"min":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.96799999999999997}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.501,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84199999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94599999999999995,"5min":1.081,"15min":1.081},"last":0.753}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90000000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.58299999999999996,"15min":0.63900000000000001},"min":{"1min":0.28000000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":2.246,"15min":2.246},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.52700000000000002,"15min":0.57299999999999995},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.48499999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.86499999999999999,"15min":0.94499999999999995},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.56599999999999995,"5min":11.913,"15min":11.913},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.083,"15min":1.3},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.63900000000000001,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.80100000000000005,"15min":0.91900000000000004},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84999999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98999999999999999,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92900000000000005,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.83299999999999996},"min":{"1min":0.59199999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.028,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.745}]},{"osd":6,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.84299999999999997},"min":{"1min":0.55100000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0580000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.58799999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.83699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":884408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48699999999999999,"15min":0.52900000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66600000000000004,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.443},{"interface":"front","average":{"1min":0.432,"5min":0.498,"15min":0.53000000000000003},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61299999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":2.29,"15min":2.29},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71399999999999997,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.57999999999999996,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95399999999999996,"5min":2.234,"15min":2.234},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80100000000000005,"15min":0.84099999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.84699999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0680000000000001,"5min":3.544,"15min":3.544},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":2.367,"15min":2.367},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98899999999999999,"5min":2.395,"15min":2.395},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.84599999999999997},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96399999999999997,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.998,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85999999999999999,"15min":0.89600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.437},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":1.369,"15min":1.369},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.501}]},{"osd":4,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.752},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1759999999999999,"5min":1.476,"15min":1.476},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.82099999999999995,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1100000000000001,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64300000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.435,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":1.155,"15min":1.155},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61299999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.751,"15min":0.77500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0449999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.292,"5min":1.304,"15min":1.304},"last":0.63}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.252,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2050000000000001,"5min":2.012,"15min":2.012},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1180000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84399999999999997,"5min":1.871,"15min":1.871},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.80400000000000005,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.877,"5min":1.375,"15min":1.375},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79500000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76600000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.5},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90800000000000003,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.40600000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90600000000000003,"5min":1.929,"15min":1.929},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84699999999999998,"5min":2.859,"15min":2.859},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96699999999999997,"5min":1.46,"15min":1.46},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.98199999999999998,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.93899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93500000000000005,"5min":1.038,"15min":1.038},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.95299999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88800000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89600000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53900000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.623,"5min":0.432,"15min":0.432},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.91400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:42:58.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:42:58.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.886+0000 7f227e3ff700 1 Processor -- start 2023-07-06T21:42:58.889 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.887+0000 7f227e3ff700 1 -- start start 2023-07-06T21:42:58.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227e3ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f22780b83c0 0x7f22780cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.890 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227e3ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f22780d3710 0x7f22780cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780cfa90 0x7f227815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227e3ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22780cff10 con 0x7f22780d3710 2023-07-06T21:42:58.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227e3ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22780d0a80 con 0x7f22780b83c0 2023-07-06T21:42:58.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22780b8790 con 0x7f22780cfa90 2023-07-06T21:42:58.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f22780d3710 0x7f22780cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f22780d3710 0x7f22780cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44446/0 (socket says 172.21.15.84:44446) 2023-07-06T21:42:58.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 -- 172.21.15.84:0/2937669335 learned_addr learned my addr 172.21.15.84:0/2937669335 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:58.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f2277fff700 1 --2- 172.21.15.84:0/2937669335 >> v2:172.21.15.107:3300/0 conn(0x7f22780b83c0 0x7f22780cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f227c99c700 1 --2- 172.21.15.84:0/2937669335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780cfa90 0x7f227815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.892 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 -- 172.21.15.84:0/2937669335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780cfa90 msgr2=0x7f227815feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 --2- 172.21.15.84:0/2937669335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780cfa90 0x7f227815feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 -- 172.21.15.84:0/2937669335 >> v2:172.21.15.107:3300/0 conn(0x7f22780b83c0 msgr2=0x7f22780cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 --2- 172.21.15.84:0/2937669335 >> v2:172.21.15.107:3300/0 conn(0x7f22780b83c0 0x7f22780cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.888+0000 7f22777fe700 1 -- 172.21.15.84:0/2937669335 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22780b4e70 con 0x7f22780d3710 2023-07-06T21:42:58.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.889+0000 7f22777fe700 1 --2- 172.21.15.84:0/2937669335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780d3710 0x7f22780cf550 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f22780cf520 tx=0x7f226c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6192d18855aad5a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:58.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.889+0000 7f2276ffd700 1 -- 172.21.15.84:0/2937669335 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00b870 con 0x7f22780d3710 2023-07-06T21:42:58.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.889+0000 7f2276ffd700 1 -- 172.21.15.84:0/2937669335 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c00b9d0 con 0x7f22780d3710 2023-07-06T21:42:58.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.889+0000 7f2276ffd700 1 -- 172.21.15.84:0/2937669335 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c00bd60 con 0x7f22780d3710 2023-07-06T21:42:58.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.889+0000 7f227e3ff700 1 -- 172.21.15.84:0/2937669335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780d3710 msgr2=0x7f22780cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.889+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2937669335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780d3710 0x7f22780cf550 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f22780cf520 tx=0x7f226c009bd0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 -- 172.21.15.84:0/2937669335 shutdown_connections 2023-07-06T21:42:58.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2937669335 >> v2:172.21.15.107:3300/0 conn(0x7f22780b83c0 0x7f22780cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2937669335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780cfa90 0x7f227815feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.896 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2937669335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780d3710 0x7f22780cf550 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 -- 172.21.15.84:0/2937669335 >> 172.21.15.84:0/2937669335 conn(0x7f227806b350 msgr2=0x7f227806b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:58.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 -- 172.21.15.84:0/2937669335 shutdown_connections 2023-07-06T21:42:58.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.890+0000 7f227e3ff700 1 -- 172.21.15.84:0/2937669335 wait complete. 2023-07-06T21:42:58.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.892+0000 7f227e3ff700 1 Processor -- start 2023-07-06T21:42:58.897 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.892+0000 7f227e3ff700 1 -- start start 2023-07-06T21:42:58.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.893+0000 7f227e3ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780b83c0 0x7f2278151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.899 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.893+0000 7f227e3ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 0x7f227815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.893+0000 7f227e3ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22780d3710 0x7f22780d1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.893+0000 7f227e3ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278151570 con 0x7f22780cfa90 2023-07-06T21:42:58.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.893+0000 7f227e3ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227815c9d0 con 0x7f22780d3710 2023-07-06T21:42:58.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.893+0000 7f227e3ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227815cc70 con 0x7f22780b83c0 2023-07-06T21:42:58.900 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 0x7f227815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 0x7f227815dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44452/0 (socket says 172.21.15.84:44452) 2023-07-06T21:42:58.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 -- 172.21.15.84:0/2089200515 learned_addr learned my addr 172.21.15.84:0/2089200515 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:58.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f2277fff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780b83c0 0x7f2278151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f227c99c700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22780d3710 0x7f22780d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.901 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 -- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780b83c0 msgr2=0x7f2278151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780b83c0 0x7f2278151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 -- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22780d3710 msgr2=0x7f22780d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:58.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22780d3710 0x7f22780d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:58.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 -- 172.21.15.84:0/2089200515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c009550 con 0x7f22780cfa90 2023-07-06T21:42:58.902 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.894+0000 7f22777fe700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 0x7f227815dc50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f226c000c00 tx=0x7f226c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:58.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.895+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c004310 con 0x7f22780cfa90 2023-07-06T21:42:58.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.895+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c01e070 con 0x7f22780cfa90 2023-07-06T21:42:58.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.895+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c0036a0 con 0x7f22780cfa90 2023-07-06T21:42:58.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.895+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227815cf50 con 0x7f22780cfa90 2023-07-06T21:42:58.903 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.895+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227815e190 con 0x7f22780cfa90 2023-07-06T21:42:58.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.896+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227815d750 con 0x7f22780cfa90 2023-07-06T21:42:58.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.897+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f226c020030 con 0x7f22780cfa90 2023-07-06T21:42:58.904 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.898+0000 7f2274ff9700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f226406ace0 0x7f226406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:58.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.898+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f226c08a050 con 0x7f22780cfa90 2023-07-06T21:42:58.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.898+0000 7f2277fff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f226406ace0 0x7f226406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:58.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.898+0000 7f2277fff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f226406ace0 0x7f226406d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22780b4bd0 tx=0x7f2268008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:58.905 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:58.902+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f226c0bc310 con 0x7f22780cfa90 2023-07-06T21:42:59.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.095+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22780b5430 con 0x7f226406ace0 2023-07-06T21:42:59.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.098+0000 7f2274ff9700 1 -- 172.21.15.84:0/2089200515 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72642 (secure 0 0 0) 0x7f22680076d0 con 0x7f226406ace0 2023-07-06T21:42:59.101 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:59.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f226406ace0 msgr2=0x7f226406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f226406ace0 0x7f226406d190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f22780b4bd0 tx=0x7f2268008040 comp rx=0 tx=0).stop 2023-07-06T21:42:59.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 msgr2=0x7f227815dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 0x7f227815dc50 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f226c000c00 tx=0x7f226c004750 comp rx=0 tx=0).stop 2023-07-06T21:42:59.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 shutdown_connections 2023-07-06T21:42:59.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f226406ace0 0x7f226406d190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22780d3710 0x7f22780d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22780b83c0 0x7f2278151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 --2- 172.21.15.84:0/2089200515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f22780cfa90 0x7f227815dc50 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 >> 172.21.15.84:0/2089200515 conn(0x7f227806b350 msgr2=0x7f22780b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:59.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 shutdown_connections 2023-07-06T21:42:59.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.101+0000 7f227e3ff700 1 -- 172.21.15.84:0/2089200515 wait complete. 2023-07-06T21:42:59.106 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:42:59.121 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2023-07-06T21:42:58.179873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221972,"kb_used_data":6356,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754752748,"statfs":{"total":773094113280,"available":772866813952,"internally_reserved":0,"allocated":6508544,"data_stored":3510599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14363,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000966"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.578034+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:42:56.578034+0000","last_peered":"2023-07-06T21:42:56.578034+0000","last_clean":"2023-07-06T21:42:56.578034+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:42:56.578034+0000","last_undegraded":"2023-07-06T21:42:56.578034+0000","last_fullsized":"2023-07-06T21:42:56.578034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:02:13.288211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128555+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:42:56.128555+0000","last_peered":"2023-07-06T21:42:56.128555+0000","last_clean":"2023-07-06T21:42:56.128555+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:42:56.128555+0000","last_undegraded":"2023-07-06T21:42:56.128555+0000","last_fullsized":"2023-07-06T21:42:56.128555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:57.209997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.837111+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:42:56.837111+0000","last_peered":"2023-07-06T21:42:56.837111+0000","last_clean":"2023-07-06T21:42:56.837111+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:42:56.837111+0000","last_undegraded":"2023-07-06T21:42:56.837111+0000","last_fullsized":"2023-07-06T21:42:56.837111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:27:05.384819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.130367+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:42:56.130367+0000","last_peered":"2023-07-06T21:42:56.130367+0000","last_clean":"2023-07-06T21:42:56.130367+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:42:56.130367+0000","last_undegraded":"2023-07-06T21:42:56.130367+0000","last_fullsized":"2023-07-06T21:42:56.130367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:23:36.326578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141735+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:42:57.141735+0000","last_peered":"2023-07-06T21:42:57.141735+0000","last_clean":"2023-07-06T21:42:57.141735+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:42:57.141735+0000","last_undegraded":"2023-07-06T21:42:57.141735+0000","last_fullsized":"2023-07-06T21:42:57.141735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:20:39.183176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141546+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:42:57.141546+0000","last_peered":"2023-07-06T21:42:57.141546+0000","last_clean":"2023-07-06T21:42:57.141546+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:42:57.141546+0000","last_undegraded":"2023-07-06T21:42:57.141546+0000","last_fullsized":"2023-07-06T21:42:57.141546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:01:48.302621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.140311+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:42:57.140311+0000","last_peered":"2023-07-06T21:42:57.140311+0000","last_clean":"2023-07-06T21:42:57.140311+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:42:57.140311+0000","last_undegraded":"2023-07-06T21:42:57.140311+0000","last_fullsized":"2023-07-06T21:42:57.140311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:28.126913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.576736+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:42:56.576736+0000","last_peered":"2023-07-06T21:42:56.576736+0000","last_clean":"2023-07-06T21:42:56.576736+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:42:56.576736+0000","last_undegraded":"2023-07-06T21:42:56.576736+0000","last_fullsized":"2023-07-06T21:42:56.576736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:02:07.005730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128278+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:42:56.128278+0000","last_peered":"2023-07-06T21:42:56.128278+0000","last_clean":"2023-07-06T21:42:56.128278+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:42:56.128278+0000","last_undegraded":"2023-07-06T21:42:56.128278+0000","last_fullsized":"2023-07-06T21:42:56.128278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:44.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1253376,"data_stored":879509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.53900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.101,"5min":1.28,"15min":1.28},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.995,"15min":0.995},"last":0.751}]},{"osd":1,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.80600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89500000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69699999999999995,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.872,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.972,"15min":1.972},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.379,"15min":1.379},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79500000000000004,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85199999999999998,"5min":3.569,"15min":3.569},"last":0.872},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.76200000000000001,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.75800000000000001,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99199999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.80000000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82999999999999996,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88500000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.85299999999999998},"min":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.96799999999999997}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.501,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84199999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94599999999999995,"5min":1.081,"15min":1.081},"last":0.753}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90000000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.58299999999999996,"15min":0.63900000000000001},"min":{"1min":0.28000000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":2.246,"15min":2.246},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.52700000000000002,"15min":0.57299999999999995},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.48499999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.86499999999999999,"15min":0.94499999999999995},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.56599999999999995,"5min":11.913,"15min":11.913},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.083,"15min":1.3},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.63900000000000001,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.80100000000000005,"15min":0.91900000000000004},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84999999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98999999999999999,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92900000000000005,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.83299999999999996},"min":{"1min":0.59199999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.028,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.745}]},{"osd":6,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.84299999999999997},"min":{"1min":0.55100000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0580000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.58799999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.83699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":884408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48699999999999999,"15min":0.52900000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66600000000000004,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.443},{"interface":"front","average":{"1min":0.432,"5min":0.498,"15min":0.53000000000000003},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61299999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":2.29,"15min":2.29},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71399999999999997,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.57999999999999996,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95399999999999996,"5min":2.234,"15min":2.234},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80100000000000005,"15min":0.84099999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.84699999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0680000000000001,"5min":3.544,"15min":3.544},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":2.367,"15min":2.367},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98899999999999999,"5min":2.395,"15min":2.395},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.84599999999999997},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96399999999999997,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.998,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85999999999999999,"15min":0.89600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.437},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":1.369,"15min":1.369},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.501}]},{"osd":4,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.752},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1759999999999999,"5min":1.476,"15min":1.476},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.82099999999999995,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1100000000000001,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64300000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.435,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":1.155,"15min":1.155},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61299999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.751,"15min":0.77500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0449999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.292,"5min":1.304,"15min":1.304},"last":0.63}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.252,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2050000000000001,"5min":2.012,"15min":2.012},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1180000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84399999999999997,"5min":1.871,"15min":1.871},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.80400000000000005,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.877,"5min":1.375,"15min":1.375},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79500000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76600000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.5},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90800000000000003,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.40600000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90600000000000003,"5min":1.929,"15min":1.929},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84699999999999998,"5min":2.859,"15min":2.859},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96699999999999997,"5min":1.46,"15min":1.46},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.98199999999999998,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.93899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93500000000000005,"5min":1.038,"15min":1.038},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.95299999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88800000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89600000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53900000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.623,"5min":0.432,"15min":0.432},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.91400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:42:59.124 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-07-06T21:42:59.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:42:59.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.315+0000 7fba281ff700 1 Processor -- start 2023-07-06T21:42:59.318 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.315+0000 7fba281ff700 1 -- start start 2023-07-06T21:42:59.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.316+0000 7fba281ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba20166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.316+0000 7fba281ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200d0990 con 0x7fba200b8530 2023-07-06T21:42:59.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba281ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200cd9d0 con 0x7fba200d3580 2023-07-06T21:42:59.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba281ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200d3af0 con 0x7fba200d1580 2023-07-06T21:42:59.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba25f9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fba200b8530 0x7fba200cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60006/0 (socket says 172.21.15.84:60006) 2023-07-06T21:42:59.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba25f9b700 1 -- 172.21.15.84:0/565880343 learned_addr learned my addr 172.21.15.84:0/565880343 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:59.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba2679c700 1 --2- 172.21.15.84:0/565880343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba20166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790766165 0 0) 0x7fba200d0990 con 0x7fba200b8530 2023-07-06T21:42:59.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba0c0035b0 con 0x7fba200b8530 2023-07-06T21:42:59.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784796974 0 0) 0x7fba200cd9d0 con 0x7fba200d3580 2023-07-06T21:42:59.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba200d0990 con 0x7fba200d3580 2023-07-06T21:42:59.322 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3221696972 0 0) 0x7fba0c0035b0 con 0x7fba200b8530 2023-07-06T21:42:59.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba2679c700 1 -- 172.21.15.84:0/565880343 >> v1:172.21.15.107:6789/0 conn(0x7fba200d3580 legacy=0x7fba200d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba2679c700 1 -- 172.21.15.84:0/565880343 >> v1:172.21.15.84:6789/0 conn(0x7fba200b8530 legacy=0x7fba200cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.323 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.317+0000 7fba2679c700 1 -- 172.21.15.84:0/565880343 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba200cd410 con 0x7fba200d1580 2023-07-06T21:42:59.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.318+0000 7fba2679c700 1 --2- 172.21.15.84:0/565880343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba20166ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba18009dc0 tx=0x7fba1800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433646c433e80c32 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:59.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.318+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800e070 con 0x7fba200d1580 2023-07-06T21:42:59.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.318+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba18003a20 con 0x7fba200d1580 2023-07-06T21:42:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.318+0000 7fba24f99700 1 -- 172.21.15.84:0/565880343 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c0017f0 con 0x7fba200d1580 2023-07-06T21:42:59.327 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 -- 172.21.15.84:0/565880343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 msgr2=0x7fba20166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 --2- 172.21.15.84:0/565880343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba20166ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fba18009dc0 tx=0x7fba1800b330 comp rx=0 tx=0).stop 2023-07-06T21:42:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 -- 172.21.15.84:0/565880343 shutdown_connections 2023-07-06T21:42:59.328 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 --2- 172.21.15.84:0/565880343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba20166ff0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 -- 172.21.15.84:0/565880343 >> 172.21.15.84:0/565880343 conn(0x7fba200715c0 msgr2=0x7fba20071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:59.329 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 -- 172.21.15.84:0/565880343 shutdown_connections 2023-07-06T21:42:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.319+0000 7fba281ff700 1 -- 172.21.15.84:0/565880343 wait complete. 2023-07-06T21:42:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.320+0000 7fba281ff700 1 Processor -- start 2023-07-06T21:42:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 -- start start 2023-07-06T21:42:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba200b8530 0x7fba20150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.330 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba200cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 0x7fba2015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c7bb0 con 0x7fba200d3580 2023-07-06T21:42:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba201f1a50 con 0x7fba200b8530 2023-07-06T21:42:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.321+0000 7fba281ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c7720 con 0x7fba200d1580 2023-07-06T21:42:59.331 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 0x7fba2015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 0x7fba2015a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44460/0 (socket says 172.21.15.84:44460) 2023-07-06T21:42:59.332 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 -- 172.21.15.84:0/2545887271 learned_addr learned my addr 172.21.15.84:0/2545887271 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 -- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 msgr2=0x7fba200cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba25f9b700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba200b8530 0x7fba20150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2579a700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba200cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba200cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.333 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 -- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba200b8530 msgr2=0x7fba20150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba200b8530 0x7fba20150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 -- 172.21.15.84:0/2545887271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18009a70 con 0x7fba200d3580 2023-07-06T21:42:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba25f9b700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba200b8530 0x7fba20150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:59.334 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2579a700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba200cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.322+0000 7fba2679c700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 0x7fba2015a780 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba1800b370 tx=0x7fba18004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.323+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba180046b0 con 0x7fba200d3580 2023-07-06T21:42:59.335 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.323+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba18004810 con 0x7fba200d3580 2023-07-06T21:42:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.323+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200cbc90 con 0x7fba200d3580 2023-07-06T21:42:59.336 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.323+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1801d670 con 0x7fba200d3580 2023-07-06T21:42:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.323+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba200c6df0 con 0x7fba200d3580 2023-07-06T21:42:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.324+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200c7890 con 0x7fba200d3580 2023-07-06T21:42:59.337 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.325+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fba1801d7d0 con 0x7fba200d3580 2023-07-06T21:42:59.338 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.326+0000 7fba167fc700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba0406ace0 0x7fba0406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.339 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.326+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fba1808b330 con 0x7fba200d3580 2023-07-06T21:42:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.326+0000 7fba25f9b700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba0406ace0 0x7fba0406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.327+0000 7fba25f9b700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba0406ace0 0x7fba0406d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fba1c001360 tx=0x7fba1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:59.340 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.329+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fba180566e0 con 0x7fba200d3580 2023-07-06T21:42:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.523+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fba200b4fc0 con 0x7fba0406ace0 2023-07-06T21:42:59.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.526+0000 7fba167fc700 1 -- 172.21.15.84:0/2545887271 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72642 (secure 0 0 0) 0x7fba1c007e20 con 0x7fba0406ace0 2023-07-06T21:42:59.529 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba0406ace0 msgr2=0x7fba0406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba0406ace0 0x7fba0406d190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fba1c001360 tx=0x7fba1c009040 comp rx=0 tx=0).stop 2023-07-06T21:42:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 msgr2=0x7fba2015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 0x7fba2015a780 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fba1800b370 tx=0x7fba18004ad0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 shutdown_connections 2023-07-06T21:42:59.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fba0406ace0 0x7fba0406d190 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fba200d3580 0x7fba2015a780 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba200b8530 0x7fba20150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 --2- 172.21.15.84:0/2545887271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba200d1580 0x7fba200cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 >> 172.21.15.84:0/2545887271 conn(0x7fba200715c0 msgr2=0x7fba200d1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:59.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 shutdown_connections 2023-07-06T21:42:59.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.529+0000 7fba281ff700 1 -- 172.21.15.84:0/2545887271 wait complete. 2023-07-06T21:42:59.534 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:42:59.549 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2023-07-06T21:42:58.179873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":196,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221972,"kb_used_data":6356,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754752748,"statfs":{"total":773094113280,"available":772866813952,"internally_reserved":0,"allocated":6508544,"data_stored":3510599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14363,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000966"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.578034+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:42:56.578034+0000","last_peered":"2023-07-06T21:42:56.578034+0000","last_clean":"2023-07-06T21:42:56.578034+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:42:56.578034+0000","last_undegraded":"2023-07-06T21:42:56.578034+0000","last_fullsized":"2023-07-06T21:42:56.578034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:02:13.288211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128555+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:42:56.128555+0000","last_peered":"2023-07-06T21:42:56.128555+0000","last_clean":"2023-07-06T21:42:56.128555+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:42:56.128555+0000","last_undegraded":"2023-07-06T21:42:56.128555+0000","last_fullsized":"2023-07-06T21:42:56.128555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:57.209997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.837111+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:42:56.837111+0000","last_peered":"2023-07-06T21:42:56.837111+0000","last_clean":"2023-07-06T21:42:56.837111+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:42:56.837111+0000","last_undegraded":"2023-07-06T21:42:56.837111+0000","last_fullsized":"2023-07-06T21:42:56.837111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:27:05.384819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.130367+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:42:56.130367+0000","last_peered":"2023-07-06T21:42:56.130367+0000","last_clean":"2023-07-06T21:42:56.130367+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:42:56.130367+0000","last_undegraded":"2023-07-06T21:42:56.130367+0000","last_fullsized":"2023-07-06T21:42:56.130367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:23:36.326578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141735+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:42:57.141735+0000","last_peered":"2023-07-06T21:42:57.141735+0000","last_clean":"2023-07-06T21:42:57.141735+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:42:57.141735+0000","last_undegraded":"2023-07-06T21:42:57.141735+0000","last_fullsized":"2023-07-06T21:42:57.141735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:20:39.183176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141546+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:42:57.141546+0000","last_peered":"2023-07-06T21:42:57.141546+0000","last_clean":"2023-07-06T21:42:57.141546+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:42:57.141546+0000","last_undegraded":"2023-07-06T21:42:57.141546+0000","last_fullsized":"2023-07-06T21:42:57.141546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:01:48.302621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.140311+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:42:57.140311+0000","last_peered":"2023-07-06T21:42:57.140311+0000","last_clean":"2023-07-06T21:42:57.140311+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:42:57.140311+0000","last_undegraded":"2023-07-06T21:42:57.140311+0000","last_fullsized":"2023-07-06T21:42:57.140311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:28.126913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.576736+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:42:56.576736+0000","last_peered":"2023-07-06T21:42:56.576736+0000","last_clean":"2023-07-06T21:42:56.576736+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:42:56.576736+0000","last_undegraded":"2023-07-06T21:42:56.576736+0000","last_fullsized":"2023-07-06T21:42:56.576736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:02:07.005730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128278+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:42:56.128278+0000","last_peered":"2023-07-06T21:42:56.128278+0000","last_clean":"2023-07-06T21:42:56.128278+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:42:56.128278+0000","last_undegraded":"2023-07-06T21:42:56.128278+0000","last_fullsized":"2023-07-06T21:42:56.128278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:44.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1253376,"data_stored":879509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.53900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.101,"5min":1.28,"15min":1.28},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.995,"15min":0.995},"last":0.751}]},{"osd":1,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.80600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89500000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69699999999999995,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.872,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.972,"15min":1.972},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.379,"15min":1.379},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.61799999999999999}]},{"osd":6,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79500000000000004,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.71999999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574896,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85199999999999998,"5min":3.569,"15min":3.569},"last":0.872},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.83499999999999996}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.76200000000000001,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.98199999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.75800000000000001,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99199999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.80000000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82999999999999996,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.78200000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88500000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.85299999999999998},"min":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.96799999999999997}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.501,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84199999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.622},{"interface":"front","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94599999999999995,"5min":1.081,"15min":1.081},"last":0.753}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90000000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.58299999999999996,"15min":0.63900000000000001},"min":{"1min":0.28000000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":2.246,"15min":2.246},"last":0.47799999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.52700000000000002,"15min":0.57299999999999995},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.48499999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.44},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.86499999999999999,"15min":0.94499999999999995},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.56599999999999995,"5min":11.913,"15min":11.913},"last":0.503}]},{"osd":3,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.083,"15min":1.3},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.63900000000000001,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.80100000000000005,"15min":0.91900000000000004},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84999999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98999999999999999,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92900000000000005,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.83299999999999996},"min":{"1min":0.59199999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.028,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.745}]},{"osd":6,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.84299999999999997},"min":{"1min":0.55100000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0580000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.80100000000000005}]},{"osd":7,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.58799999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.83699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":884408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48699999999999999,"15min":0.52900000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66600000000000004,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.443},{"interface":"front","average":{"1min":0.432,"5min":0.498,"15min":0.53000000000000003},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61299999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":2.29,"15min":2.29},"last":0.46899999999999997}]},{"osd":3,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71399999999999997,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.57999999999999996,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.51700000000000002}]},{"osd":4,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95399999999999996,"5min":2.234,"15min":2.234},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80100000000000005,"15min":0.84099999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.84699999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0680000000000001,"5min":3.544,"15min":3.544},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":2.367,"15min":2.367},"last":0.79100000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98899999999999999,"5min":2.395,"15min":2.395},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.84599999999999997},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96399999999999997,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.998,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85999999999999999,"15min":0.89600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.32600000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.437},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":1.369,"15min":1.369},"last":0.38800000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.501}]},{"osd":4,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.752},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1759999999999999,"5min":1.476,"15min":1.476},"last":0.64800000000000002}]},{"osd":6,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.82099999999999995,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1100000000000001,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.746}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64300000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.435,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":1.155,"15min":1.155},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61299999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.751,"15min":0.77500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0449999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.292,"5min":1.304,"15min":1.304},"last":0.63}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.252,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.59799999999999998}]},{"osd":6,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2050000000000001,"5min":2.012,"15min":2.012},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1180000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574896,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":659456,"data_stored":289141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84399999999999997,"5min":1.871,"15min":1.871},"last":0.59499999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.80400000000000005,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.877,"5min":1.375,"15min":1.375},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79500000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76600000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.5},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90800000000000003,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.40600000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90600000000000003,"5min":1.929,"15min":1.929},"last":0.88200000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84699999999999998,"5min":2.859,"15min":2.859},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96699999999999997,"5min":1.46,"15min":1.46},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.98199999999999998,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.93899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574896,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93500000000000005,"5min":1.038,"15min":1.038},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.95299999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88800000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.88300000000000001}]},{"osd":4,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89600000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53900000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.80400000000000005}]},{"osd":6,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.623,"5min":0.432,"15min":0.432},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.91400000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:42:59.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-06T21:42:59.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.742+0000 7f7f3a1ff700 1 Processor -- start 2023-07-06T21:42:59.745 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.743+0000 7f7f3a1ff700 1 -- start start 2023-07-06T21:42:59.749 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f3a1ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7f3415c6a0 0x7f7f3406c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f3a1ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7f340b3010 0x7f7f3406afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.750 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f3a1ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f3406b8a0 con 0x7f7f340b2470 2023-07-06T21:42:59.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f3a1ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3415ca70 con 0x7f7f3415c6a0 2023-07-06T21:42:59.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f3a1ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f3406bb80 con 0x7f7f340b3010 2023-07-06T21:42:59.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f33fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7f340b2470 0x7f7f340cb040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60010/0 (socket says 172.21.15.84:60010) 2023-07-06T21:42:59.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f33fff700 1 -- 172.21.15.84:0/2517204777 learned_addr learned my addr 172.21.15.84:0/2517204777 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:42:59.751 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f337fe700 1 --2- 172.21.15.84:0/2517204777 >> v2:172.21.15.84:3300/0 conn(0x7f7f3415c6a0 0x7f7f3406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f32ffd700 1 --2- 172.21.15.84:0/2517204777 >> v2:172.21.15.107:3300/0 conn(0x7f7f340b3010 0x7f7f3406afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f337fe700 1 -- 172.21.15.84:0/2517204777 >> v2:172.21.15.107:3300/0 conn(0x7f7f340b3010 msgr2=0x7f7f3406afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:42:59.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f337fe700 1 --2- 172.21.15.84:0/2517204777 >> v2:172.21.15.107:3300/0 conn(0x7f7f340b3010 0x7f7f3406afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f337fe700 1 -- 172.21.15.84:0/2517204777 >> v1:172.21.15.84:6789/0 conn(0x7f7f340b2470 legacy=0x7f7f340cb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.752 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f337fe700 1 -- 172.21.15.84:0/2517204777 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3406be60 con 0x7f7f3415c6a0 2023-07-06T21:42:59.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.744+0000 7f7f327fc700 1 -- 172.21.15.84:0/2517204777 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897059654 0 0) 0x7f7f3406b8a0 con 0x7f7f340b2470 2023-07-06T21:42:59.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.745+0000 7f7f337fe700 1 --2- 172.21.15.84:0/2517204777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3415c6a0 0x7f7f3406c940 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7f340d0950 tx=0x7f7f24009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438e0aabb68623f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:59.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.745+0000 7f7f327fc700 1 -- 172.21.15.84:0/2517204777 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24011830 con 0x7f7f3415c6a0 2023-07-06T21:42:59.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.745+0000 7f7f327fc700 1 -- 172.21.15.84:0/2517204777 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f24011990 con 0x7f7f3415c6a0 2023-07-06T21:42:59.753 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.745+0000 7f7f327fc700 1 -- 172.21.15.84:0/2517204777 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f24011d20 con 0x7f7f3415c6a0 2023-07-06T21:42:59.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.745+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/2517204777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3415c6a0 msgr2=0x7f7f3406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.745+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/2517204777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3415c6a0 0x7f7f3406c940 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7f340d0950 tx=0x7f7f24009b20 comp rx=0 tx=0).stop 2023-07-06T21:42:59.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.746+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/2517204777 shutdown_connections 2023-07-06T21:42:59.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.746+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/2517204777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f3415c6a0 0x7f7f3406c940 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.754 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.746+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/2517204777 >> v2:172.21.15.107:3300/0 conn(0x7f7f340b3010 0x7f7f3406afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.746+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/2517204777 >> 172.21.15.84:0/2517204777 conn(0x7f7f3406e640 msgr2=0x7f7f3406ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:59.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.746+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/2517204777 shutdown_connections 2023-07-06T21:42:59.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.746+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/2517204777 wait complete. 2023-07-06T21:42:59.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.747+0000 7f7f3a1ff700 1 Processor -- start 2023-07-06T21:42:59.755 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.747+0000 7f7f3a1ff700 1 -- start start 2023-07-06T21:42:59.757 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.748+0000 7f7f3a1ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f340b2470 0x7f7f340c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.748+0000 7f7f3a1ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 0x7f7f340c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.748+0000 7f7f3a1ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3415c6a0 0x7f7f340c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.748+0000 7f7f3a1ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340c7830 con 0x7f7f340b3010 2023-07-06T21:42:59.758 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.748+0000 7f7f3a1ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340ccfd0 con 0x7f7f340b2470 2023-07-06T21:42:59.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.748+0000 7f7f3a1ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f340c6cf0 con 0x7f7f3415c6a0 2023-07-06T21:42:59.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 0x7f7f340c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 0x7f7f340c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44488/0 (socket says 172.21.15.84:44488) 2023-07-06T21:42:59.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 -- 172.21.15.84:0/572786529 learned_addr learned my addr 172.21.15.84:0/572786529 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:42:59.759 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f337fe700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f340b2470 0x7f7f340c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 -- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3415c6a0 msgr2=0x7f7f340c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f33fff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3415c6a0 0x7f7f340c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3415c6a0 0x7f7f340c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 -- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f340b2470 msgr2=0x7f7f340c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.760 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f340b2470 0x7f7f340c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 -- 172.21.15.84:0/572786529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240094a0 con 0x7f7f340b3010 2023-07-06T21:42:59.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f33fff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3415c6a0 0x7f7f340c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:42:59.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.749+0000 7f7f32ffd700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 0x7f7f340c4780 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9b0 tx=0x7f7f2800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:59.761 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.750+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28009710 con 0x7f7f340b3010 2023-07-06T21:42:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.750+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2800c850 con 0x7f7f340b3010 2023-07-06T21:42:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.750+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2800cbc0 con 0x7f7f340b3010 2023-07-06T21:42:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.750+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f340c6f90 con 0x7f7f340b3010 2023-07-06T21:42:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.750+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f340c7550 con 0x7f7f340b3010 2023-07-06T21:42:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.751+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f340b4880 con 0x7f7f340b3010 2023-07-06T21:42:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.752+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7f2800cd20 con 0x7f7f340b3010 2023-07-06T21:42:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.753+0000 7f7f1bfff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7f2006ace0 0x7f7f2006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:42:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.753+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7f28089370 con 0x7f7f340b3010 2023-07-06T21:42:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.753+0000 7f7f337fe700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7f2006ace0 0x7f7f2006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:42:59.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.753+0000 7f7f337fe700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7f2006ace0 0x7f7f2006d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f240090b0 tx=0x7f7f2401c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:42:59.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.757+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7f28058130 con 0x7f7f340b3010 2023-07-06T21:42:59.958 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.956+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f3404e240 con 0x7f7f340b3010 2023-07-06T21:42:59.959 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.957+0000 7f7f1bfff700 1 -- 172.21.15.84:0/572786529 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+14770 (secure 0 0 0) 0x7f7f28057cc0 con 0x7f7f340b3010 2023-07-06T21:42:59.959 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:42:59.960 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":56,"fsid":"c2762452-5842-4ccc-b0b9-8dbf6ac718c9","created":"2023-07-06T21:38:58.919599+0000","modified":"2023-07-06T21:42:56.120573+0000","last_up_change":"2023-07-06T21:39:11.278518+0000","last_in_change":"2023-07-06T21:39:02.333069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-06T21:39:12.175836+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-07-06T21:39:18.713799+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"972346d6-6871-4d63-9180-095dd8ba1fed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6801","nonce":3639070743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6803","nonce":3639070743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6807","nonce":3639070743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":3639070743},{"type":"v1","addr":"172.21.15.84:6805","nonce":3639070743}]},"public_addr":"172.21.15.84:6801/3639070743","cluster_addr":"172.21.15.84:6803/3639070743","heartbeat_back_addr":"172.21.15.84:6807/3639070743","heartbeat_front_addr":"172.21.15.84:6805/3639070743","state":["exists","up"]},{"osd":1,"uuid":"42a1f42a-4cec-4d2f-8943-efd17495b1c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6817","nonce":1216611025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6819","nonce":1216611025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6823","nonce":1216611025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":1216611025},{"type":"v1","addr":"172.21.15.84:6821","nonce":1216611025}]},"public_addr":"172.21.15.84:6817/1216611025","cluster_addr":"172.21.15.84:6819/1216611025","heartbeat_back_addr":"172.21.15.84:6823/1216611025","heartbeat_front_addr":"172.21.15.84:6821/1216611025","state":["exists","up"]},{"osd":2,"uuid":"e354fea8-9ad5-42e4-a409-658253c8b4e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6809","nonce":3137136770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6811","nonce":3137136770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6815","nonce":3137136770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":3137136770},{"type":"v1","addr":"172.21.15.84:6813","nonce":3137136770}]},"public_addr":"172.21.15.84:6809/3137136770","cluster_addr":"172.21.15.84:6811/3137136770","heartbeat_back_addr":"172.21.15.84:6815/3137136770","heartbeat_front_addr":"172.21.15.84:6813/3137136770","state":["exists","up"]},{"osd":3,"uuid":"a531fbd4-2b94-4043-b9dd-6c6c106f5321","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6825","nonce":3293194179}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6827","nonce":3293194179}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6831","nonce":3293194179}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":3293194179},{"type":"v1","addr":"172.21.15.84:6829","nonce":3293194179}]},"public_addr":"172.21.15.84:6825/3293194179","cluster_addr":"172.21.15.84:6827/3293194179","heartbeat_back_addr":"172.21.15.84:6831/3293194179","heartbeat_front_addr":"172.21.15.84:6829/3293194179","state":["exists","up"]},{"osd":4,"uuid":"4c8b22f3-ae3f-42af-80b7-63029b5f82f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6801","nonce":1667769091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6803","nonce":1667769091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6807","nonce":1667769091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1667769091},{"type":"v1","addr":"172.21.15.107:6805","nonce":1667769091}]},"public_addr":"172.21.15.107:6801/1667769091","cluster_addr":"172.21.15.107:6803/1667769091","heartbeat_back_addr":"172.21.15.107:6807/1667769091","heartbeat_front_addr":"172.21.15.107:6805/1667769091","state":["exists","up"]},{"osd":5,"uuid":"a66f2d6d-f11e-4f56-a186-44033dc51676","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6809","nonce":3858042795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6811","nonce":3858042795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6815","nonce":3858042795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3858042795},{"type":"v1","addr":"172.21.15.107:6813","nonce":3858042795}]},"public_addr":"172.21.15.107:6809/3858042795","cluster_addr":"172.21.15.107:6811/3858042795","heartbeat_back_addr":"172.21.15.107:6815/3858042795","heartbeat_front_addr":"172.21.15.107:6813/3858042795","state":["exists","up"]},{"osd":6,"uuid":"c5fc70b9-b03f-4319-8d63-f793ebd4e6ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6817","nonce":2301301903}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6819","nonce":2301301903}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6823","nonce":2301301903}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2301301903},{"type":"v1","addr":"172.21.15.107:6821","nonce":2301301903}]},"public_addr":"172.21.15.107:6817/2301301903","cluster_addr":"172.21.15.107:6819/2301301903","heartbeat_back_addr":"172.21.15.107:6823/2301301903","heartbeat_front_addr":"172.21.15.107:6821/2301301903","state":["exists","up"]},{"osd":7,"uuid":"ca2f0297-5efb-4620-83a1-8f137a4a0718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6825","nonce":440998536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6827","nonce":440998536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6831","nonce":440998536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":440998536},{"type":"v1","addr":"172.21.15.107:6829","nonce":440998536}]},"public_addr":"172.21.15.107:6825/440998536","cluster_addr":"172.21.15.107:6827/440998536","heartbeat_back_addr":"172.21.15.107:6831/440998536","heartbeat_front_addr":"172.21.15.107:6829/440998536","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.380943+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.689677+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.626820+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.638294+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.714142+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.598534+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.692876+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-06T21:39:07.887544+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6834/2341549811":"2023-07-07T21:42:52.522543+0000","172.21.15.107:6832/2341549811":"2023-07-07T21:42:52.522543+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-06T21:42:59.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7f2006ace0 msgr2=0x7f7f2006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7f2006ace0 0x7f7f2006d190 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7f240090b0 tx=0x7f7f2401c040 comp rx=0 tx=0).stop 2023-07-06T21:42:59.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 msgr2=0x7f7f340c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:42:59.962 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 0x7f7f340c4780 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7f2800e9b0 tx=0x7f7f2800ed70 comp rx=0 tx=0).stop 2023-07-06T21:42:59.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 shutdown_connections 2023-07-06T21:42:59.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7f2006ace0 0x7f7f2006d190 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f3415c6a0 0x7f7f340c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.963 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f340b2470 0x7f7f340c67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 --2- 172.21.15.84:0/572786529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7f340b3010 0x7f7f340c4780 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:42:59.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 >> 172.21.15.84:0/572786529 conn(0x7f7f3406e640 msgr2=0x7f7f340caff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:42:59.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 shutdown_connections 2023-07-06T21:42:59.964 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:42:59.960+0000 7f7f3a1ff700 1 -- 172.21.15.84:0/572786529 wait complete. 2023-07-06T21:43:00.980 INFO:tasks.ceph:Scrubbing osd.0 2023-07-06T21:43:00.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:01.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.186+0000 7fd3321ff700 1 Processor -- start 2023-07-06T21:43:01.188 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.186+0000 7fd3321ff700 1 -- start start 2023-07-06T21:43:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd3321ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fd32c160a10 0x7fd32c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd3321ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd32c06a5a0 0x7fd32c0c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd3321ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32c164480 con 0x7fd32c160640 2023-07-06T21:43:01.189 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd3321ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c165100 con 0x7fd32c160a10 2023-07-06T21:43:01.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd3321ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd32c0c97d0 con 0x7fd32c06a5a0 2023-07-06T21:43:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd32b7fe700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fd32c160640 0x7fd32c069e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60012/0 (socket says 172.21.15.84:60012) 2023-07-06T21:43:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd32b7fe700 1 -- 172.21.15.84:0/2080929675 learned_addr learned my addr 172.21.15.84:0/2080929675 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd323fff700 1 --2- 172.21.15.84:0/2080929675 >> v2:172.21.15.84:3300/0 conn(0x7fd32c160a10 0x7fd32c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.187+0000 7fd32bfff700 1 --2- 172.21.15.84:0/2080929675 >> v2:172.21.15.107:3300/0 conn(0x7fd32c06a5a0 0x7fd32c0c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd32affd700 1 -- 172.21.15.84:0/2080929675 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139188396 0 0) 0x7fd32c164480 con 0x7fd32c160640 2023-07-06T21:43:01.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd32affd700 1 -- 172.21.15.84:0/2080929675 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3180035b0 con 0x7fd32c160640 2023-07-06T21:43:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd323fff700 1 -- 172.21.15.84:0/2080929675 >> v2:172.21.15.107:3300/0 conn(0x7fd32c06a5a0 msgr2=0x7fd32c0c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd323fff700 1 --2- 172.21.15.84:0/2080929675 >> v2:172.21.15.107:3300/0 conn(0x7fd32c06a5a0 0x7fd32c0c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd323fff700 1 -- 172.21.15.84:0/2080929675 >> v1:172.21.15.84:6789/0 conn(0x7fd32c160640 legacy=0x7fd32c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd323fff700 1 -- 172.21.15.84:0/2080929675 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c164760 con 0x7fd32c160a10 2023-07-06T21:43:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd323fff700 1 --2- 172.21.15.84:0/2080929675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0c8d50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd32c06ad50 tx=0x7fd314009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7e76d5895eee40 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd32affd700 1 -- 172.21.15.84:0/2080929675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314011830 con 0x7fd32c160a10 2023-07-06T21:43:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd32affd700 1 -- 172.21.15.84:0/2080929675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314011990 con 0x7fd32c160a10 2023-07-06T21:43:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.188+0000 7fd32affd700 1 -- 172.21.15.84:0/2080929675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314011d20 con 0x7fd32c160a10 2023-07-06T21:43:01.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 -- 172.21.15.84:0/2080929675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 msgr2=0x7fd32c0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 --2- 172.21.15.84:0/2080929675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0c8d50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd32c06ad50 tx=0x7fd314009b20 comp rx=0 tx=0).stop 2023-07-06T21:43:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 -- 172.21.15.84:0/2080929675 shutdown_connections 2023-07-06T21:43:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 --2- 172.21.15.84:0/2080929675 >> v2:172.21.15.107:3300/0 conn(0x7fd32c06a5a0 0x7fd32c0c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 --2- 172.21.15.84:0/2080929675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0c8d50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 -- 172.21.15.84:0/2080929675 >> 172.21.15.84:0/2080929675 conn(0x7fd32c0cbe10 msgr2=0x7fd32c0ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 -- 172.21.15.84:0/2080929675 shutdown_connections 2023-07-06T21:43:01.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.189+0000 7fd3321ff700 1 -- 172.21.15.84:0/2080929675 wait complete. 2023-07-06T21:43:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.191+0000 7fd3321ff700 1 Processor -- start 2023-07-06T21:43:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.191+0000 7fd3321ff700 1 -- start start 2023-07-06T21:43:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd3321ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd32c06a5a0 0x7fd32c0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd3321ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd32c160640 0x7fd32c15eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd3321ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd3321ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c15fb10 con 0x7fd32c160a10 2023-07-06T21:43:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd3321ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c15f090 con 0x7fd32c160640 2023-07-06T21:43:01.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd3321ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32c0aeda0 con 0x7fd32c06a5a0 2023-07-06T21:43:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44516/0 (socket says 172.21.15.84:44516) 2023-07-06T21:43:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 -- 172.21.15.84:0/2693682213 learned_addr learned my addr 172.21.15.84:0/2693682213 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd323fff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd32c160640 0x7fd32c15eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32b7fe700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd32c06a5a0 0x7fd32c0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 -- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd32c06a5a0 msgr2=0x7fd32c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd32c06a5a0 0x7fd32c0b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 -- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd32c160640 msgr2=0x7fd32c15eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd32c160640 0x7fd32c15eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32bfff700 1 -- 172.21.15.84:0/2693682213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3140094a0 con 0x7fd32c160a10 2023-07-06T21:43:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.192+0000 7fd32b7fe700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd32c06a5a0 0x7fd32c0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.193+0000 7fd32bfff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0af230 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd324009dd0 tx=0x7fd32400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.193+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324009c40 con 0x7fd32c160a10 2023-07-06T21:43:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.193+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32400f040 con 0x7fd32c160a10 2023-07-06T21:43:01.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.193+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd324014480 con 0x7fd32c160a10 2023-07-06T21:43:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.193+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32c1eaf10 con 0x7fd32c160a10 2023-07-06T21:43:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.193+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32c0af770 con 0x7fd32c160a10 2023-07-06T21:43:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.194+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd32c164d20 con 0x7fd32c160a10 2023-07-06T21:43:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.195+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fd3240145e0 con 0x7fd32c160a10 2023-07-06T21:43:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd328ff9700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd30c06ace0 0x7fd30c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd324089ac0 con 0x7fd32c160a10 2023-07-06T21:43:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd32b7fe700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd30c06ace0 0x7fd30c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd328ff9700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7fd30c071150 0x7fd30c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 --> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd30c073c30 con 0x7fd30c071150 2023-07-06T21:43:01.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32408e2a0 con 0x7fd32c160a10 2023-07-06T21:43:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.196+0000 7fd323fff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7fd30c071150 0x7fd30c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.197+0000 7fd32b7fe700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd30c06ace0 0x7fd30c06d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0011e0 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.197+0000 7fd323fff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7fd30c071150 0x7fd30c073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.198+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== osd.0 v2:172.21.15.84:6800/3639070743 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd30c073c30 con 0x7fd30c071150 2023-07-06T21:43:01.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.212+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 --> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd32c164a40 con 0x7fd30c071150 2023-07-06T21:43:01.215 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd328ff9700 1 -- 172.21.15.84:0/2693682213 <== osd.0 v2:172.21.15.84:6800/3639070743 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd314011630 con 0x7fd30c071150 2023-07-06T21:43:01.215 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:01.216 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:01.216 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:01.216 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7fd30c071150 msgr2=0x7fd30c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6800/3639070743,v1:172.21.15.84:6801/3639070743] conn(0x7fd30c071150 0x7fd30c073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd30c06ace0 msgr2=0x7fd30c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fd30c06ace0 0x7fd30c06d190 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd31c0011e0 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2023-07-06T21:43:01.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 msgr2=0x7fd32c0af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 --2- 172.21.15.84:0/2693682213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd32c160a10 0x7fd32c0af230 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd324009dd0 tx=0x7fd32400c3a0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd32bfff700 1 -- 172.21.15.84:0/2693682213 reap_dead start 2023-07-06T21:43:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 shutdown_connections 2023-07-06T21:43:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 >> 172.21.15.84:0/2693682213 conn(0x7fd32c0cbe10 msgr2=0x7fd32c0b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 shutdown_connections 2023-07-06T21:43:01.218 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.213+0000 7fd3321ff700 1 -- 172.21.15.84:0/2693682213 wait complete. 2023-07-06T21:43:01.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-07-06T21:43:01.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.424+0000 7f99c9dff700 1 Processor -- start 2023-07-06T21:43:01.426 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.424+0000 7f99c9dff700 1 -- start start 2023-07-06T21:43:01.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c9dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99c40d3710 0x7f99c40cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c9dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c40d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c9dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40cd910 con 0x7f99c40b83c0 2023-07-06T21:43:01.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c9dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c40cff10 con 0x7f99c40d3710 2023-07-06T21:43:01.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c9dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40d0a80 con 0x7f99c40cfa90 2023-07-06T21:43:01.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c2ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99c40d3710 0x7f99c40cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c2ffd700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f99c40d3710 0x7f99c40cf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44518/0 (socket says 172.21.15.84:44518) 2023-07-06T21:43:01.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.425+0000 7f99c2ffd700 1 -- 172.21.15.84:0/482062807 learned_addr learned my addr 172.21.15.84:0/482062807 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:01.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c3fff700 1 --2- 172.21.15.84:0/482062807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c40d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c2ffd700 1 -- 172.21.15.84:0/482062807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 msgr2=0x7f99c40d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:01.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c2ffd700 1 --2- 172.21.15.84:0/482062807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c40d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c2ffd700 1 -- 172.21.15.84:0/482062807 >> v1:172.21.15.107:6789/0 conn(0x7f99c40b83c0 legacy=0x7f99c40cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c2ffd700 1 -- 172.21.15.84:0/482062807 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c40b8790 con 0x7f99c40d3710 2023-07-06T21:43:01.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c2ffd700 1 --2- 172.21.15.84:0/482062807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40d3710 0x7f99c40cf550 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99b80098f0 tx=0x7f99b8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5ac537852ac40e server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c27fc700 1 -- 172.21.15.84:0/482062807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800b8a0 con 0x7f99c40d3710 2023-07-06T21:43:01.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c27fc700 1 -- 172.21.15.84:0/482062807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b800ba00 con 0x7f99c40d3710 2023-07-06T21:43:01.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.426+0000 7f99c27fc700 1 -- 172.21.15.84:0/482062807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800bd90 con 0x7f99c40d3710 2023-07-06T21:43:01.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 -- 172.21.15.84:0/482062807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40d3710 msgr2=0x7f99c40cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 --2- 172.21.15.84:0/482062807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40d3710 0x7f99c40cf550 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99b80098f0 tx=0x7f99b8009c00 comp rx=0 tx=0).stop 2023-07-06T21:43:01.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 -- 172.21.15.84:0/482062807 shutdown_connections 2023-07-06T21:43:01.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 --2- 172.21.15.84:0/482062807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c40d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 --2- 172.21.15.84:0/482062807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40d3710 0x7f99c40cf550 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 -- 172.21.15.84:0/482062807 >> 172.21.15.84:0/482062807 conn(0x7f99c406b350 msgr2=0x7f99c406b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 -- 172.21.15.84:0/482062807 shutdown_connections 2023-07-06T21:43:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.427+0000 7f99c9dff700 1 -- 172.21.15.84:0/482062807 wait complete. 2023-07-06T21:43:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.429+0000 7f99c9dff700 1 Processor -- start 2023-07-06T21:43:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.429+0000 7f99c9dff700 1 -- start start 2023-07-06T21:43:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c9dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 0x7f99c41502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c9dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c4150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c9dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c40d3710 0x7f99c4150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c9dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40cc650 con 0x7f99c40b83c0 2023-07-06T21:43:01.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c9dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40cc930 con 0x7f99c40d3710 2023-07-06T21:43:01.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c9dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40ca6c0 con 0x7f99c40cfa90 2023-07-06T21:43:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 0x7f99c41502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 0x7f99c41502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44524/0 (socket says 172.21.15.84:44524) 2023-07-06T21:43:01.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 -- 172.21.15.84:0/1768346515 learned_addr learned my addr 172.21.15.84:0/1768346515 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:01.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c2ffd700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c4150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c3fff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c40d3710 0x7f99c4150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 -- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 msgr2=0x7f99c4150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c4150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 -- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c40d3710 msgr2=0x7f99c4150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c40d3710 0x7f99c4150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.430+0000 7f99c37fe700 1 -- 172.21.15.84:0/1768346515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8009550 con 0x7f99c40b83c0 2023-07-06T21:43:01.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.431+0000 7f99c37fe700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 0x7f99c41502d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c450 tx=0x7f99b400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.431+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40052d0 con 0x7f99c40b83c0 2023-07-06T21:43:01.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.431+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4005430 con 0x7f99c40b83c0 2023-07-06T21:43:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.431+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c40ca9a0 con 0x7f99c40b83c0 2023-07-06T21:43:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.431+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40038c0 con 0x7f99c40b83c0 2023-07-06T21:43:01.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.431+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c40cc290 con 0x7f99c40b83c0 2023-07-06T21:43:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.432+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c40cc420 con 0x7f99c40b83c0 2023-07-06T21:43:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.433+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f99b4003a20 con 0x7f99c40b83c0 2023-07-06T21:43:01.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.434+0000 7f99abfff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f99b006ace0 0x7f99b006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.434+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f99b40883a0 con 0x7f99c40b83c0 2023-07-06T21:43:01.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.434+0000 7f99c2ffd700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f99b006ace0 0x7f99b006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.434+0000 7f99c2ffd700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f99b006ace0 0x7f99b006d190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99c40d3b20 tx=0x7f99b8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.438+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f99b40536d0 con 0x7f99c40b83c0 2023-07-06T21:43:01.634 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.632+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c40b5150 con 0x7f99b006ace0 2023-07-06T21:43:01.635 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.632+0000 7f99abfff700 1 -- 172.21.15.84:0/1768346515 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f99b800bb70 con 0x7f99b006ace0 2023-07-06T21:43:01.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.634+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f99b006ace0 msgr2=0x7f99b006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.634+0000 7f99c9dff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f99b006ace0 0x7f99b006d190 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f99c40d3b20 tx=0x7f99b8008030 comp rx=0 tx=0).stop 2023-07-06T21:43:01.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.634+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 msgr2=0x7f99c41502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.637 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.634+0000 7f99c9dff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 0x7f99c41502d0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f99b400c450 tx=0x7f99b400c760 comp rx=0 tx=0).stop 2023-07-06T21:43:01.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 shutdown_connections 2023-07-06T21:43:01.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f99b006ace0 0x7f99b006d190 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99c40d3710 0x7f99c4150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.638 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99c40b83c0 0x7f99c41502d0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 --2- 172.21.15.84:0/1768346515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99c40cfa90 0x7f99c4150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 >> 172.21.15.84:0/1768346515 conn(0x7f99c406b350 msgr2=0x7f99c40b6690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:01.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 shutdown_connections 2023-07-06T21:43:01.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.635+0000 7f99c9dff700 1 -- 172.21.15.84:0/1768346515 wait complete. 2023-07-06T21:43:01.640 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 0 to deep-scrub 2023-07-06T21:43:01.655 INFO:tasks.ceph:Scrubbing osd.1 2023-07-06T21:43:01.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:01.851 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.849+0000 7fbec19ff700 1 Processor -- start 2023-07-06T21:43:01.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.849+0000 7fbec19ff700 1 -- start start 2023-07-06T21:43:01.852 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.850+0000 7fbec19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc0d0ea0 0x7fbebc0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbec19ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc0ceef0 con 0x7fbebc160ac0 2023-07-06T21:43:01.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbec19ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbebc0cf450 con 0x7fbebc15fee0 2023-07-06T21:43:01.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbec19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0cf120 con 0x7fbebc0d0ea0 2023-07-06T21:43:01.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeb3fff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbebc160ac0 0x7fbebc0cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60022/0 (socket says 172.21.15.84:60022) 2023-07-06T21:43:01.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeb3fff700 1 -- 172.21.15.84:0/352907592 learned_addr learned my addr 172.21.15.84:0/352907592 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:01.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbebaffd700 1 --2- 172.21.15.84:0/352907592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc0d0ea0 0x7fbebc0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643573879 0 0) 0x7fbebc0ceef0 con 0x7fbebc160ac0 2023-07-06T21:43:01.861 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbea80035b0 con 0x7fbebc160ac0 2023-07-06T21:43:01.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661346209 0 0) 0x7fbebc0cf450 con 0x7fbebc15fee0 2023-07-06T21:43:01.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbebc0ceef0 con 0x7fbebc15fee0 2023-07-06T21:43:01.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbebaffd700 1 -- 172.21.15.84:0/352907592 >> v1:172.21.15.107:6789/0 conn(0x7fbebc15fee0 legacy=0x7fbebc0d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549795202 0 0) 0x7fbea80035b0 con 0x7fbebc160ac0 2023-07-06T21:43:01.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbebaffd700 1 -- 172.21.15.84:0/352907592 >> v1:172.21.15.84:6789/0 conn(0x7fbebc160ac0 legacy=0x7fbebc0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.851+0000 7fbebaffd700 1 -- 172.21.15.84:0/352907592 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc0cd420 con 0x7fbebc0d0ea0 2023-07-06T21:43:01.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.852+0000 7fbebaffd700 1 --2- 172.21.15.84:0/352907592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc0d0ea0 0x7fbebc0d1280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009850 tx=0x7fbeac009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25765537e1d0b244 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.852+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4001670 con 0x7fbebc0d0ea0 2023-07-06T21:43:01.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.852+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac005c30 con 0x7fbebc0d0ea0 2023-07-06T21:43:01.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.852+0000 7fbeba7fc700 1 -- 172.21.15.84:0/352907592 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac004030 con 0x7fbebc0d0ea0 2023-07-06T21:43:01.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.852+0000 7fbec19ff700 1 -- 172.21.15.84:0/352907592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc0d0ea0 msgr2=0x7fbebc0d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.852+0000 7fbec19ff700 1 --2- 172.21.15.84:0/352907592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc0d0ea0 0x7fbebc0d1280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009850 tx=0x7fbeac009b60 comp rx=0 tx=0).stop 2023-07-06T21:43:01.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.853+0000 7fbec19ff700 1 -- 172.21.15.84:0/352907592 shutdown_connections 2023-07-06T21:43:01.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.853+0000 7fbec19ff700 1 --2- 172.21.15.84:0/352907592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc0d0ea0 0x7fbebc0d1280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.853+0000 7fbec19ff700 1 -- 172.21.15.84:0/352907592 >> 172.21.15.84:0/352907592 conn(0x7fbebc0715c0 msgr2=0x7fbebc071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:01.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.853+0000 7fbec19ff700 1 -- 172.21.15.84:0/352907592 shutdown_connections 2023-07-06T21:43:01.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.853+0000 7fbec19ff700 1 -- 172.21.15.84:0/352907592 wait complete. 2023-07-06T21:43:01.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.854+0000 7fbec19ff700 1 Processor -- start 2023-07-06T21:43:01.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.854+0000 7fbec19ff700 1 -- start start 2023-07-06T21:43:01.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.855+0000 7fbec19ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15fee0 0x7fbebc0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.855+0000 7fbec19ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbebc160ac0 0x7fbebc0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.855+0000 7fbec19ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc1502d0 0x7fbebc0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.855+0000 7fbec19ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0cb010 con 0x7fbebc15fee0 2023-07-06T21:43:01.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.855+0000 7fbec19ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc0c9390 con 0x7fbebc160ac0 2023-07-06T21:43:01.871 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.855+0000 7fbec19ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbebc150a60 con 0x7fbebc1502d0 2023-07-06T21:43:01.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15fee0 0x7fbebc0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbeb3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbebc160ac0 0x7fbebc0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15fee0 0x7fbebc0cb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44538/0 (socket says 172.21.15.84:44538) 2023-07-06T21:43:01.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 -- 172.21.15.84:0/1406639438 learned_addr learned my addr 172.21.15.84:0/1406639438 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:01.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebb7fe700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc1502d0 0x7fbebc0c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 -- 172.21.15.84:0/1406639438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc1502d0 msgr2=0x7fbebc0c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbebc1502d0 0x7fbebc0c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 -- 172.21.15.84:0/1406639438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbebc160ac0 msgr2=0x7fbebc0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbebc160ac0 0x7fbebc0c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 -- 172.21.15.84:0/1406639438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac009430 con 0x7fbebc15fee0 2023-07-06T21:43:01.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbeb3fff700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbebc160ac0 0x7fbebc0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:43:01.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.856+0000 7fbebaffd700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15fee0 0x7fbebc0cb4a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0d3da0 tx=0x7fbeac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.857+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac003ab0 con 0x7fbebc15fee0 2023-07-06T21:43:01.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.857+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac003c10 con 0x7fbebc15fee0 2023-07-06T21:43:01.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.857+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbebc0ca400 con 0x7fbebc15fee0 2023-07-06T21:43:01.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.857+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac01d430 con 0x7fbebc15fee0 2023-07-06T21:43:01.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.857+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbebc0ca9c0 con 0x7fbebc15fee0 2023-07-06T21:43:01.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.858+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbebc0cfff0 con 0x7fbebc15fee0 2023-07-06T21:43:01.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.859+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbeac01d590 con 0x7fbebc15fee0 2023-07-06T21:43:01.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb8ff9700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe9c06ace0 0x7fbe9c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbeac08a110 con 0x7fbebc15fee0 2023-07-06T21:43:01.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb3fff700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe9c06ace0 0x7fbe9c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb8ff9700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7fbe9c071150 0x7fbe9c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:01.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 --> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe9c073c30 con 0x7fbe9c071150 2023-07-06T21:43:01.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeac08c6f0 con 0x7fbebc15fee0 2023-07-06T21:43:01.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbebb7fe700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7fbe9c071150 0x7fbe9c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:01.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.860+0000 7fbeb3fff700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe9c06ace0 0x7fbe9c06d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0b8a40 tx=0x7fbea4007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.861+0000 7fbebb7fe700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7fbe9c071150 0x7fbe9c073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:01.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.861+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== osd.1 v2:172.21.15.84:6816/1216611025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbe9c073c30 con 0x7fbe9c071150 2023-07-06T21:43:01.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.876+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 --> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbebc0cd700 con 0x7fbe9c071150 2023-07-06T21:43:01.882 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbeb8ff9700 1 -- 172.21.15.84:0/1406639438 <== osd.1 v2:172.21.15.84:6816/1216611025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fbeb4001fe0 con 0x7fbe9c071150 2023-07-06T21:43:01.882 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:01.883 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:01.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7fbe9c071150 msgr2=0x7fbe9c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6816/1216611025,v1:172.21.15.84:6817/1216611025] conn(0x7fbe9c071150 0x7fbe9c073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe9c06ace0 msgr2=0x7fbe9c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbe9c06ace0 0x7fbe9c06d190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0b8a40 tx=0x7fbea4007de0 comp rx=0 tx=0).stop 2023-07-06T21:43:01.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15fee0 msgr2=0x7fbebc0cb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:01.886 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 --2- 172.21.15.84:0/1406639438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbebc15fee0 0x7fbebc0cb4a0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0d3da0 tx=0x7fbeac004750 comp rx=0 tx=0).stop 2023-07-06T21:43:01.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbebb7fe700 1 -- 172.21.15.84:0/1406639438 reap_dead start 2023-07-06T21:43:01.887 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 shutdown_connections 2023-07-06T21:43:01.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 >> 172.21.15.84:0/1406639438 conn(0x7fbebc0715c0 msgr2=0x7fbebc0d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:01.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.877+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 shutdown_connections 2023-07-06T21:43:01.888 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:01.878+0000 7fbec19ff700 1 -- 172.21.15.84:0/1406639438 wait complete. 2023-07-06T21:43:01.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-07-06T21:43:02.092 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.090+0000 7f5a32dff700 1 Processor -- start 2023-07-06T21:43:02.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.090+0000 7f5a32dff700 1 -- start start 2023-07-06T21:43:02.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.091+0000 7f5a32dff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5a2c0b83c0 0x7f5a2c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.091+0000 7f5a32dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5a2c0d3710 0x7f5a2c0cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.091+0000 7f5a32dff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c0d0a80 con 0x7f5a2c0cfa90 2023-07-06T21:43:02.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.091+0000 7f5a32dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2c0cff10 con 0x7f5a2c0d3710 2023-07-06T21:43:02.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.091+0000 7f5a32dff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2c0b8790 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a3139c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f5a2c0cfa90 0x7f5a2c15feb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60038/0 (socket says 172.21.15.84:60038) 2023-07-06T21:43:02.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2bfff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f5a2c0d3710 0x7f5a2c0cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a3139c700 1 -- 172.21.15.84:0/4093846643 learned_addr learned my addr 172.21.15.84:0/4093846643 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:02.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a30b9b700 1 --2- 172.21.15.84:0/4093846643 >> v2:172.21.15.107:3300/0 conn(0x7f5a2c0b83c0 0x7f5a2c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2b7fe700 1 -- 172.21.15.84:0/4093846643 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736155412 0 0) 0x7f5a2c0d0a80 con 0x7f5a2c0cfa90 2023-07-06T21:43:02.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2b7fe700 1 -- 172.21.15.84:0/4093846643 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a1c0035b0 con 0x7f5a2c0cfa90 2023-07-06T21:43:02.097 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2bfff700 1 -- 172.21.15.84:0/4093846643 >> v2:172.21.15.107:3300/0 conn(0x7f5a2c0b83c0 msgr2=0x7f5a2c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:02.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2bfff700 1 --2- 172.21.15.84:0/4093846643 >> v2:172.21.15.107:3300/0 conn(0x7f5a2c0b83c0 0x7f5a2c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2bfff700 1 -- 172.21.15.84:0/4093846643 >> v1:172.21.15.84:6789/0 conn(0x7f5a2c0cfa90 legacy=0x7f5a2c15feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.092+0000 7f5a2bfff700 1 -- 172.21.15.84:0/4093846643 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c0b4e70 con 0x7f5a2c0d3710 2023-07-06T21:43:02.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a2bfff700 1 --2- 172.21.15.84:0/4093846643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0d3710 0x7f5a2c0cf550 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0cf520 tx=0x7f5a18009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d033bf42ab63ea server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a2b7fe700 1 -- 172.21.15.84:0/4093846643 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18011830 con 0x7f5a2c0d3710 2023-07-06T21:43:02.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a2b7fe700 1 -- 172.21.15.84:0/4093846643 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a18011990 con 0x7f5a2c0d3710 2023-07-06T21:43:02.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a2b7fe700 1 -- 172.21.15.84:0/4093846643 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18011d20 con 0x7f5a2c0d3710 2023-07-06T21:43:02.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 -- 172.21.15.84:0/4093846643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0d3710 msgr2=0x7f5a2c0cf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 --2- 172.21.15.84:0/4093846643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0d3710 0x7f5a2c0cf550 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0cf520 tx=0x7f5a18009b20 comp rx=0 tx=0).stop 2023-07-06T21:43:02.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 -- 172.21.15.84:0/4093846643 shutdown_connections 2023-07-06T21:43:02.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 --2- 172.21.15.84:0/4093846643 >> v2:172.21.15.107:3300/0 conn(0x7f5a2c0b83c0 0x7f5a2c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 --2- 172.21.15.84:0/4093846643 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0d3710 0x7f5a2c0cf550 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 -- 172.21.15.84:0/4093846643 >> 172.21.15.84:0/4093846643 conn(0x7f5a2c06b350 msgr2=0x7f5a2c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:02.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.093+0000 7f5a32dff700 1 -- 172.21.15.84:0/4093846643 shutdown_connections 2023-07-06T21:43:02.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.094+0000 7f5a32dff700 1 -- 172.21.15.84:0/4093846643 wait complete. 2023-07-06T21:43:02.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.095+0000 7f5a32dff700 1 Processor -- start 2023-07-06T21:43:02.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.095+0000 7f5a32dff700 1 -- start start 2023-07-06T21:43:02.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a32dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 0x7f5a2c1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a32dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a2c0cfa90 0x7f5a2c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a32dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a2c0d3710 0x7f5a2c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a32dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c154610 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a32dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c157d20 con 0x7f5a2c0cfa90 2023-07-06T21:43:02.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a32dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c15ea80 con 0x7f5a2c0d3710 2023-07-06T21:43:02.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a30b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 0x7f5a2c1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a30b9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 0x7f5a2c1540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44568/0 (socket says 172.21.15.84:44568) 2023-07-06T21:43:02.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.096+0000 7f5a30b9b700 1 -- 172.21.15.84:0/3556721675 learned_addr learned my addr 172.21.15.84:0/3556721675 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:02.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a2bfff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a2c0cfa90 0x7f5a2c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a30b9b700 1 -- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a2c0d3710 msgr2=0x7f5a2c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a30b9b700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a2c0d3710 0x7f5a2c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a30b9b700 1 -- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a2c0cfa90 msgr2=0x7f5a2c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a30b9b700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a2c0cfa90 0x7f5a2c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a30b9b700 1 -- 172.21.15.84:0/3556721675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180094a0 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a30b9b700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 0x7f5a2c1540d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000e870 tx=0x7f5a2000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20009590 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2000ee70 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.107 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c15ed60 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2000ca50 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.097+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c15f320 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.099+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c15adb0 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.099+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f5a2000cbb0 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.100+0000 7f5a297fa700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5a1006ace0 0x7f5a1006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.100+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5a20089100 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.101+0000 7f5a2bfff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5a1006ace0 0x7f5a1006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.101+0000 7f5a2bfff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5a1006ace0 0x7f5a1006d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0d3b20 tx=0x7f5a1801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.104+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f5a20057fd0 con 0x7f5a2c0b83c0 2023-07-06T21:43:02.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.303+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a2c0b5430 con 0x7f5a1006ace0 2023-07-06T21:43:02.309 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.303+0000 7f5a297fa700 1 -- 172.21.15.84:0/3556721675 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5a18011630 con 0x7f5a1006ace0 2023-07-06T21:43:02.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.305+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5a1006ace0 msgr2=0x7f5a1006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.305+0000 7f5a32dff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5a1006ace0 0x7f5a1006d190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0d3b20 tx=0x7f5a1801c040 comp rx=0 tx=0).stop 2023-07-06T21:43:02.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.305+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 msgr2=0x7f5a2c1540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.305+0000 7f5a32dff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 0x7f5a2c1540d0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000e870 tx=0x7f5a2000ec30 comp rx=0 tx=0).stop 2023-07-06T21:43:02.310 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.305+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 shutdown_connections 2023-07-06T21:43:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f5a1006ace0 0x7f5a1006d190 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a2c0d3710 0x7f5a2c15e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5a2c0b83c0 0x7f5a2c1540d0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 --2- 172.21.15.84:0/3556721675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a2c0cfa90 0x7f5a2c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.311 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 >> 172.21.15.84:0/3556721675 conn(0x7f5a2c06b350 msgr2=0x7f5a2c0b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:02.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 shutdown_connections 2023-07-06T21:43:02.312 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.306+0000 7f5a32dff700 1 -- 172.21.15.84:0/3556721675 wait complete. 2023-07-06T21:43:02.312 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 1 to deep-scrub 2023-07-06T21:43:02.326 INFO:tasks.ceph:Scrubbing osd.2 2023-07-06T21:43:02.327 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.521+0000 7ff1503ff700 1 Processor -- start 2023-07-06T21:43:02.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.521+0000 7ff1503ff700 1 -- start start 2023-07-06T21:43:02.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff1503ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff14806a330 0x7ff1480c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff1503ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1480c8d50 con 0x7ff148069e90 2023-07-06T21:43:02.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff1503ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1480c9310 con 0x7ff14815d7b0 2023-07-06T21:43:02.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff1503ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14815db80 con 0x7ff14806a330 2023-07-06T21:43:02.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff14d99a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff148069e90 0x7ff1480c97a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60040/0 (socket says 172.21.15.84:60040) 2023-07-06T21:43:02.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff14d99a700 1 -- 172.21.15.84:0/2154140388 learned_addr learned my addr 172.21.15.84:0/2154140388 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:02.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.522+0000 7ff14e99c700 1 --2- 172.21.15.84:0/2154140388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff14806a330 0x7ff1480c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142709181 0 0) 0x7ff1480c8d50 con 0x7ff148069e90 2023-07-06T21:43:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1340035b0 con 0x7ff148069e90 2023-07-06T21:43:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3607555328 0 0) 0x7ff1480c9310 con 0x7ff14815d7b0 2023-07-06T21:43:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1480c8d50 con 0x7ff14815d7b0 2023-07-06T21:43:02.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14e99c700 1 -- 172.21.15.84:0/2154140388 >> v1:172.21.15.107:6789/0 conn(0x7ff14815d7b0 legacy=0x7ff14815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14e99c700 1 -- 172.21.15.84:0/2154140388 >> v1:172.21.15.84:6789/0 conn(0x7ff148069e90 legacy=0x7ff1480c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14e99c700 1 -- 172.21.15.84:0/2154140388 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14806a7b0 con 0x7ff14806a330 2023-07-06T21:43:02.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14e99c700 1 --2- 172.21.15.84:0/2154140388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff14806a330 0x7ff1480c9eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff140009dc0 tx=0x7ff14000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6684cf5798b9df1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.523+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000e070 con 0x7ff14806a330 2023-07-06T21:43:02.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14000bb20 con 0x7ff14806a330 2023-07-06T21:43:02.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff14d199700 1 -- 172.21.15.84:0/2154140388 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff140012430 con 0x7ff14806a330 2023-07-06T21:43:02.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 -- 172.21.15.84:0/2154140388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff14806a330 msgr2=0x7ff1480c9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 --2- 172.21.15.84:0/2154140388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff14806a330 0x7ff1480c9eb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff140009dc0 tx=0x7ff14000b330 comp rx=0 tx=0).stop 2023-07-06T21:43:02.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 -- 172.21.15.84:0/2154140388 shutdown_connections 2023-07-06T21:43:02.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 --2- 172.21.15.84:0/2154140388 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff14806a330 0x7ff1480c9eb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 -- 172.21.15.84:0/2154140388 >> 172.21.15.84:0/2154140388 conn(0x7ff1480cbe10 msgr2=0x7ff1480ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:02.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 -- 172.21.15.84:0/2154140388 shutdown_connections 2023-07-06T21:43:02.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.524+0000 7ff1503ff700 1 -- 172.21.15.84:0/2154140388 wait complete. 2023-07-06T21:43:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.526+0000 7ff1503ff700 1 Processor -- start 2023-07-06T21:43:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.526+0000 7ff1503ff700 1 -- start start 2023-07-06T21:43:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff1503ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff148069e90 0x7ff148152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff1503ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff14806a330 0x7ff1481f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff1503ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff14815d7b0 0x7ff148155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff1503ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1480cc650 con 0x7ff14815d7b0 2023-07-06T21:43:02.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff1503ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff148153190 con 0x7ff14806a330 2023-07-06T21:43:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff1503ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1480cc2d0 con 0x7ff148069e90 2023-07-06T21:43:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff14e99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff14815d7b0 0x7ff148155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff14e99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff14815d7b0 0x7ff148155b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44572/0 (socket says 172.21.15.84:44572) 2023-07-06T21:43:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff14e99c700 1 -- 172.21.15.84:0/3471225533 learned_addr learned my addr 172.21.15.84:0/3471225533 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:02.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff14d99a700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff14806a330 0x7ff1481f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.527+0000 7ff14e19b700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff148069e90 0x7ff148152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff14e99c700 1 -- 172.21.15.84:0/3471225533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff148069e90 msgr2=0x7ff148152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff14e99c700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff148069e90 0x7ff148152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff14e99c700 1 -- 172.21.15.84:0/3471225533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff14806a330 msgr2=0x7ff1481f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff14e99c700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff14806a330 0x7ff1481f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff14e99c700 1 -- 172.21.15.84:0/3471225533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140009a70 con 0x7ff14815d7b0 2023-07-06T21:43:02.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff14e99c700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff14815d7b0 0x7ff148155b30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff14000b370 tx=0x7ff140004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14001b810 con 0x7ff14815d7b0 2023-07-06T21:43:02.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff140020070 con 0x7ff14815d7b0 2023-07-06T21:43:02.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1481f2a00 con 0x7ff14815d7b0 2023-07-06T21:43:02.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14001bbb0 con 0x7ff14815d7b0 2023-07-06T21:43:02.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.528+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1481541d0 con 0x7ff14815d7b0 2023-07-06T21:43:02.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.529+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff14806c9b0 con 0x7ff14815d7b0 2023-07-06T21:43:02.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.530+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff14001bd10 con 0x7ff14815d7b0 2023-07-06T21:43:02.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.531+0000 7ff13effd700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff12c06ace0 0x7ff12c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff140022070 con 0x7ff14815d7b0 2023-07-06T21:43:02.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff14e19b700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff12c06ace0 0x7ff12c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff13effd700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7ff12c071150 0x7ff12c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 --> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff12c073c30 con 0x7ff12c071150 2023-07-06T21:43:02.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff14008d4e0 con 0x7ff14815d7b0 2023-07-06T21:43:02.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff14d99a700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7ff12c071150 0x7ff12c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.532+0000 7ff14e19b700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff12c06ace0 0x7ff12c06d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff1440011e0 tx=0x7ff144009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.533+0000 7ff14d99a700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7ff12c071150 0x7ff12c073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.533+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== osd.2 v2:172.21.15.84:6808/3137136770 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff12c073c30 con 0x7ff12c071150 2023-07-06T21:43:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.548+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 --> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff14806c6d0 con 0x7ff12c071150 2023-07-06T21:43:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.548+0000 7ff13effd700 1 -- 172.21.15.84:0/3471225533 <== osd.2 v2:172.21.15.84:6808/3137136770 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff138006bb0 con 0x7ff12c071150 2023-07-06T21:43:02.551 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:02.551 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:02.552 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7ff12c071150 msgr2=0x7ff12c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6808/3137136770,v1:172.21.15.84:6809/3137136770] conn(0x7ff12c071150 0x7ff12c073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff12c06ace0 msgr2=0x7ff12c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff12c06ace0 0x7ff12c06d190 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff1440011e0 tx=0x7ff144009040 comp rx=0 tx=0).stop 2023-07-06T21:43:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff14815d7b0 msgr2=0x7ff148155b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 --2- 172.21.15.84:0/3471225533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff14815d7b0 0x7ff148155b30 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff14000b370 tx=0x7ff140004ad0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff14e99c700 1 -- 172.21.15.84:0/3471225533 reap_dead start 2023-07-06T21:43:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 shutdown_connections 2023-07-06T21:43:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 >> 172.21.15.84:0/3471225533 conn(0x7ff1480cbe10 msgr2=0x7ff1480c96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 shutdown_connections 2023-07-06T21:43:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.549+0000 7ff1503ff700 1 -- 172.21.15.84:0/3471225533 wait complete. 2023-07-06T21:43:02.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-07-06T21:43:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.760+0000 7f98cc9ff700 1 Processor -- start 2023-07-06T21:43:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.760+0000 7f98cc9ff700 1 -- start start 2023-07-06T21:43:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98cc9ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f98c80d2690 0x7f98c80cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98cc9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80b59c0 0x7f98c815fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98cc9ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c80d1460 con 0x7f98c80b1fd0 2023-07-06T21:43:02.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98cc9ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c80d2530 con 0x7f98c80d2690 2023-07-06T21:43:02.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98cc9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80d1740 con 0x7f98c80b59c0 2023-07-06T21:43:02.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98c659c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f98c80b1fd0 0x7f98c80cf160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:60052/0 (socket says 172.21.15.84:60052) 2023-07-06T21:43:02.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98c659c700 1 -- 172.21.15.84:0/3934895944 learned_addr learned my addr 172.21.15.84:0/3934895944 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:02.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98bffff700 1 --2- 172.21.15.84:0/3934895944 >> v2:172.21.15.84:3300/0 conn(0x7f98c80d2690 0x7f98c80cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.761+0000 7f98c6d9d700 1 --2- 172.21.15.84:0/3934895944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80b59c0 0x7f98c815fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98bffff700 1 -- 172.21.15.84:0/3934895944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80b59c0 msgr2=0x7f98c815fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98bffff700 1 --2- 172.21.15.84:0/3934895944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80b59c0 0x7f98c815fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98bffff700 1 -- 172.21.15.84:0/3934895944 >> v1:172.21.15.84:6789/0 conn(0x7f98c80b1fd0 legacy=0x7f98c80cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.767 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98bffff700 1 -- 172.21.15.84:0/3934895944 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c80cd260 con 0x7f98c80d2690 2023-07-06T21:43:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98c5d9b700 1 -- 172.21.15.84:0/3934895944 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595265617 0 0) 0x7f98c80d1460 con 0x7f98c80b1fd0 2023-07-06T21:43:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98bffff700 1 --2- 172.21.15.84:0/3934895944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80d2690 0x7f98c80cf870 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009840 tx=0x7f98b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1988ab35d11e7b74 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98c5d9b700 1 -- 172.21.15.84:0/3934895944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011830 con 0x7f98c80d2690 2023-07-06T21:43:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98c5d9b700 1 -- 172.21.15.84:0/3934895944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0011990 con 0x7f98c80d2690 2023-07-06T21:43:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.762+0000 7f98c5d9b700 1 -- 172.21.15.84:0/3934895944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011d20 con 0x7f98c80d2690 2023-07-06T21:43:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 -- 172.21.15.84:0/3934895944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80d2690 msgr2=0x7f98c80cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/3934895944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80d2690 0x7f98c80cf870 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009840 tx=0x7f98b0009b50 comp rx=0 tx=0).stop 2023-07-06T21:43:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 -- 172.21.15.84:0/3934895944 shutdown_connections 2023-07-06T21:43:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/3934895944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80b59c0 0x7f98c815fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/3934895944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80d2690 0x7f98c80cf870 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 -- 172.21.15.84:0/3934895944 >> 172.21.15.84:0/3934895944 conn(0x7f98c806b340 msgr2=0x7f98c806b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 -- 172.21.15.84:0/3934895944 shutdown_connections 2023-07-06T21:43:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.763+0000 7f98cc9ff700 1 -- 172.21.15.84:0/3934895944 wait complete. 2023-07-06T21:43:02.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.764+0000 7f98cc9ff700 1 Processor -- start 2023-07-06T21:43:02.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.765+0000 7f98cc9ff700 1 -- start start 2023-07-06T21:43:02.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.765+0000 7f98cc9ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c80b1fd0 0x7f98c815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98cc9ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 0x7f98c8150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98cc9ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80d2690 0x7f98c8151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98cc9ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c81515d0 con 0x7f98c80b59c0 2023-07-06T21:43:02.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98cc9ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c80a94d0 con 0x7f98c80b1fd0 2023-07-06T21:43:02.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98cc9ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c815cdc0 con 0x7f98c80d2690 2023-07-06T21:43:02.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 0x7f98c8150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 0x7f98c8150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44586/0 (socket says 172.21.15.84:44586) 2023-07-06T21:43:02.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 -- 172.21.15.84:0/2702373844 learned_addr learned my addr 172.21.15.84:0/2702373844 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:02.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98c659c700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c80b1fd0 0x7f98c815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98c6d9d700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80d2690 0x7f98c8151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 -- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80d2690 msgr2=0x7f98c8151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80d2690 0x7f98c8151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 -- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c80b1fd0 msgr2=0x7f98c815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c80b1fd0 0x7f98c815ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98bffff700 1 -- 172.21.15.84:0/2702373844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00094a0 con 0x7f98c80b59c0 2023-07-06T21:43:02.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.766+0000 7f98c6d9d700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80d2690 0x7f98c8151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:02.777 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.767+0000 7f98bffff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 0x7f98c8150b50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f98b00090b0 tx=0x7f98b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.767+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b00036a0 con 0x7f98c80b59c0 2023-07-06T21:43:02.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.767+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b001f070 con 0x7f98c80b59c0 2023-07-06T21:43:02.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.767+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0003a00 con 0x7f98c80b59c0 2023-07-06T21:43:02.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.767+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c815d0a0 con 0x7f98c80b59c0 2023-07-06T21:43:02.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.767+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c815d660 con 0x7f98c80b59c0 2023-07-06T21:43:02.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.768+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c804e240 con 0x7f98c80b59c0 2023-07-06T21:43:02.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.769+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f98b0003b60 con 0x7f98c80b59c0 2023-07-06T21:43:02.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.770+0000 7f98beffd700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f98a806ace0 0x7f98a806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:02.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.770+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f98b0021070 con 0x7f98c80b59c0 2023-07-06T21:43:02.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.770+0000 7f98c659c700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f98a806ace0 0x7f98a806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:02.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.770+0000 7f98c659c700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f98a806ace0 0x7f98a806d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000f60 tx=0x7f98b8007e70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:02.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.774+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f98b0055f40 con 0x7f98c80b59c0 2023-07-06T21:43:02.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.969+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f98c80d1150 con 0x7f98a806ace0 2023-07-06T21:43:02.972 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.969+0000 7f98beffd700 1 -- 172.21.15.84:0/2702373844 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f98b80133f0 con 0x7f98a806ace0 2023-07-06T21:43:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f98a806ace0 msgr2=0x7f98a806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.974 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f98a806ace0 0x7f98a806d190 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f98b8000f60 tx=0x7f98b8007e70 comp rx=0 tx=0).stop 2023-07-06T21:43:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 msgr2=0x7f98c8150b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 0x7f98c8150b50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f98b00090b0 tx=0x7f98b0004750 comp rx=0 tx=0).stop 2023-07-06T21:43:02.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 shutdown_connections 2023-07-06T21:43:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f98a806ace0 0x7f98a806d190 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98c80d2690 0x7f98c8151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98c80b1fd0 0x7f98c815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 --2- 172.21.15.84:0/2702373844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98c80b59c0 0x7f98c8150b50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:02.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 >> 172.21.15.84:0/2702373844 conn(0x7f98c806b340 msgr2=0x7f98c80b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:02.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 shutdown_connections 2023-07-06T21:43:02.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:02.972+0000 7f98cc9ff700 1 -- 172.21.15.84:0/2702373844 wait complete. 2023-07-06T21:43:02.978 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 2 to deep-scrub 2023-07-06T21:43:02.992 INFO:tasks.ceph:Scrubbing osd.3 2023-07-06T21:43:02.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:03.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.188+0000 7f3cb3fff700 1 Processor -- start 2023-07-06T21:43:03.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.188+0000 7f3cb3fff700 1 -- start start 2023-07-06T21:43:03.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb3fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3cac160ac0 0x7f3cac0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac15fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb3fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cac0ceef0 con 0x7f3cac0d0ea0 2023-07-06T21:43:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb3fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0cf450 con 0x7f3cac160ac0 2023-07-06T21:43:03.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb3fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0cf120 con 0x7f3cac0cfe20 2023-07-06T21:43:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb1d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f3cac0d0ea0 0x7f3cac0d1280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34044/0 (socket says 172.21.15.84:34044) 2023-07-06T21:43:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.189+0000 7f3cb1d9b700 1 -- 172.21.15.84:0/1549212997 learned_addr learned my addr 172.21.15.84:0/1549212997 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb259c700 1 --2- 172.21.15.84:0/1549212997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac15fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb159a700 1 --2- 172.21.15.84:0/1549212997 >> v2:172.21.15.107:3300/0 conn(0x7f3cac160ac0 0x7f3cac0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb0d99700 1 -- 172.21.15.84:0/1549212997 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051610106 0 0) 0x7f3cac0ceef0 con 0x7f3cac0d0ea0 2023-07-06T21:43:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb0d99700 1 -- 172.21.15.84:0/1549212997 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c980035b0 con 0x7f3cac0d0ea0 2023-07-06T21:43:03.196 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb259c700 1 -- 172.21.15.84:0/1549212997 >> v2:172.21.15.107:3300/0 conn(0x7f3cac160ac0 msgr2=0x7f3cac0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:03.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb259c700 1 --2- 172.21.15.84:0/1549212997 >> v2:172.21.15.107:3300/0 conn(0x7f3cac160ac0 0x7f3cac0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb259c700 1 -- 172.21.15.84:0/1549212997 >> v1:172.21.15.84:6789/0 conn(0x7f3cac0d0ea0 legacy=0x7f3cac0d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb259c700 1 -- 172.21.15.84:0/1549212997 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0cd420 con 0x7f3cac0cfe20 2023-07-06T21:43:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb159a700 1 --2- 172.21.15.84:0/1549212997 >> v2:172.21.15.107:3300/0 conn(0x7f3cac160ac0 0x7f3cac0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:43:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.190+0000 7f3cb259c700 1 --2- 172.21.15.84:0/1549212997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac15fee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009d90 tx=0x7f3ca400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f18a7db7b083646 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb0d99700 1 -- 172.21.15.84:0/1549212997 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca400e070 con 0x7f3cac0cfe20 2023-07-06T21:43:03.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb0d99700 1 -- 172.21.15.84:0/1549212997 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca400bb20 con 0x7f3cac0cfe20 2023-07-06T21:43:03.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb0d99700 1 -- 172.21.15.84:0/1549212997 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4012430 con 0x7f3cac0cfe20 2023-07-06T21:43:03.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 -- 172.21.15.84:0/1549212997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 msgr2=0x7f3cac15fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 --2- 172.21.15.84:0/1549212997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac15fee0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009d90 tx=0x7f3ca400b330 comp rx=0 tx=0).stop 2023-07-06T21:43:03.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 -- 172.21.15.84:0/1549212997 shutdown_connections 2023-07-06T21:43:03.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 --2- 172.21.15.84:0/1549212997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac15fee0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 --2- 172.21.15.84:0/1549212997 >> v2:172.21.15.107:3300/0 conn(0x7f3cac160ac0 0x7f3cac0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 -- 172.21.15.84:0/1549212997 >> 172.21.15.84:0/1549212997 conn(0x7f3cac0715c0 msgr2=0x7f3cac071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:03.201 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 -- 172.21.15.84:0/1549212997 shutdown_connections 2023-07-06T21:43:03.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.191+0000 7f3cb3fff700 1 -- 172.21.15.84:0/1549212997 wait complete. 2023-07-06T21:43:03.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.193+0000 7f3cb3fff700 1 Processor -- start 2023-07-06T21:43:03.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.193+0000 7f3cb3fff700 1 -- start start 2023-07-06T21:43:03.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cac160ac0 0x7f3cac0b8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cac0b8880 0x7f3cac0d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb3fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b7340 con 0x7f3cac0b8880 2023-07-06T21:43:03.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb3fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b64e0 con 0x7f3cac160ac0 2023-07-06T21:43:03.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb3fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0b67c0 con 0x7f3cac0cfe20 2023-07-06T21:43:03.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cac0b8880 0x7f3cac0d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cac0b8880 0x7f3cac0d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38666/0 (socket says 172.21.15.84:38666) 2023-07-06T21:43:03.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.194+0000 7f3cb259c700 1 -- 172.21.15.84:0/2678008383 learned_addr learned my addr 172.21.15.84:0/2678008383 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:03.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb1d9b700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb159a700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cac160ac0 0x7f3cac0b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb259c700 1 -- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 msgr2=0x7f3cac152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb259c700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb259c700 1 -- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cac160ac0 msgr2=0x7f3cac0b8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb259c700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cac160ac0 0x7f3cac0b8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb259c700 1 -- 172.21.15.84:0/2678008383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0094a0 con 0x7f3cac0b8880 2023-07-06T21:43:03.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb159a700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cac160ac0 0x7f3cac0b8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:03.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb1d9b700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cac0cfe20 0x7f3cac152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:03.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb259c700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cac0b8880 0x7f3cac0d27e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4014040 tx=0x7f3ca4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4012a90 con 0x7f3cac0b8880 2023-07-06T21:43:03.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca4020070 con 0x7f3cac0b8880 2023-07-06T21:43:03.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca4009a70 con 0x7f3cac0b8880 2023-07-06T21:43:03.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4012e50 con 0x7f3cac0b8880 2023-07-06T21:43:03.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.195+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac0b6d80 con 0x7f3cac0b8880 2023-07-06T21:43:03.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.196+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3cac0cd9e0 con 0x7f3cac0b8880 2023-07-06T21:43:03.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.197+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f3ca4004040 con 0x7f3cac0b8880 2023-07-06T21:43:03.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.198+0000 7f3ca27fc700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c9006ace0 0x7f3c9006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.198+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3ca4022070 con 0x7f3cac0b8880 2023-07-06T21:43:03.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.198+0000 7f3cb1d9b700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c9006ace0 0x7f3c9006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.198+0000 7f3ca27fc700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7f3c90071150 0x7f3c90073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.198+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 --> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c90073c30 con 0x7f3c90071150 2023-07-06T21:43:03.213 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.198+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ca408d740 con 0x7f3cac0b8880 2023-07-06T21:43:03.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.199+0000 7f3cb159a700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7f3c90071150 0x7f3c90073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.199+0000 7f3cb1d9b700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c9006ace0 0x7f3c9006d190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80010e0 tx=0x7f3ca8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.199+0000 7f3cb159a700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7f3c90071150 0x7f3c90073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.214 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.200+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== osd.3 v2:172.21.15.84:6824/3293194179 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3c90073c30 con 0x7f3c90071150 2023-07-06T21:43:03.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.215+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 --> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3cac0cd700 con 0x7f3c90071150 2023-07-06T21:43:03.217 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.215+0000 7f3ca27fc700 1 -- 172.21.15.84:0/2678008383 <== osd.3 v2:172.21.15.84:6824/3293194179 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3c9c009db0 con 0x7f3c90071150 2023-07-06T21:43:03.217 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:03.218 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:03.218 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:03.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.215+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7f3c90071150 msgr2=0x7f3c90073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.219 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.215+0000 7f3cb3fff700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6824/3293194179,v1:172.21.15.84:6825/3293194179] conn(0x7f3c90071150 0x7f3c90073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c9006ace0 msgr2=0x7f3c9006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f3c9006ace0 0x7f3c9006d190 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80010e0 tx=0x7f3ca8009040 comp rx=0 tx=0).stop 2023-07-06T21:43:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cac0b8880 msgr2=0x7f3cac0d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 --2- 172.21.15.84:0/2678008383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cac0b8880 0x7f3cac0d27e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4014040 tx=0x7f3ca4004ad0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb259c700 1 -- 172.21.15.84:0/2678008383 reap_dead start 2023-07-06T21:43:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 shutdown_connections 2023-07-06T21:43:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 >> 172.21.15.84:0/2678008383 conn(0x7f3cac0715c0 msgr2=0x7f3cac15ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:03.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 shutdown_connections 2023-07-06T21:43:03.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.216+0000 7f3cb3fff700 1 -- 172.21.15.84:0/2678008383 wait complete. 2023-07-06T21:43:03.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-07-06T21:43:03.427 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.425+0000 7f96145ff700 1 Processor -- start 2023-07-06T21:43:03.428 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.426+0000 7f96145ff700 1 -- start start 2023-07-06T21:43:03.429 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f96145ff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f960c0b83c0 0x7f960c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f96145ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c0d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f96145ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960c0d0a80 con 0x7f960c0d3710 2023-07-06T21:43:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f96145ff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f960c0b8790 con 0x7f960c0b83c0 2023-07-06T21:43:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f96145ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0b4e70 con 0x7f960c0cfc60 2023-07-06T21:43:03.430 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f960c0b83c0 0x7f960c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f960c0b83c0 0x7f960c0cd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38680/0 (socket says 172.21.15.84:38680) 2023-07-06T21:43:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 -- 172.21.15.84:0/4227852525 learned_addr learned my addr 172.21.15.84:0/4227852525 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f9612b9c700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c0d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.431 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 -- 172.21.15.84:0/4227852525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 msgr2=0x7f960c0d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c0d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 -- 172.21.15.84:0/4227852525 >> v1:172.21.15.107:6789/0 conn(0x7f960c0d3710 legacy=0x7f960c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f961239b700 1 -- 172.21.15.84:0/4227852525 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c0b5150 con 0x7f960c0b83c0 2023-07-06T21:43:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.427+0000 7f9612b9c700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:43:03.432 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.428+0000 7f961239b700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0b83c0 0x7f960c0cd200 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f96080098f0 tx=0x7f9608009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4b40ebd9fbe4b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.428+0000 7f9611399700 1 -- 172.21.15.84:0/4227852525 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800b8a0 con 0x7f960c0b83c0 2023-07-06T21:43:03.436 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.428+0000 7f9611399700 1 -- 172.21.15.84:0/4227852525 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960800ba00 con 0x7f960c0b83c0 2023-07-06T21:43:03.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.428+0000 7f9611399700 1 -- 172.21.15.84:0/4227852525 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800bd90 con 0x7f960c0b83c0 2023-07-06T21:43:03.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.428+0000 7f96145ff700 1 -- 172.21.15.84:0/4227852525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0b83c0 msgr2=0x7f960c0cd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.437 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.428+0000 7f96145ff700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0b83c0 0x7f960c0cd200 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f96080098f0 tx=0x7f9608009c00 comp rx=0 tx=0).stop 2023-07-06T21:43:03.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.429+0000 7f96145ff700 1 -- 172.21.15.84:0/4227852525 shutdown_connections 2023-07-06T21:43:03.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.429+0000 7f96145ff700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c0d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.438 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.429+0000 7f96145ff700 1 --2- 172.21.15.84:0/4227852525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0b83c0 0x7f960c0cd200 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.429+0000 7f96145ff700 1 -- 172.21.15.84:0/4227852525 >> 172.21.15.84:0/4227852525 conn(0x7f960c06b350 msgr2=0x7f960c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:03.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.429+0000 7f96145ff700 1 -- 172.21.15.84:0/4227852525 shutdown_connections 2023-07-06T21:43:03.439 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.429+0000 7f96145ff700 1 -- 172.21.15.84:0/4227852525 wait complete. 2023-07-06T21:43:03.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.430+0000 7f96145ff700 1 Processor -- start 2023-07-06T21:43:03.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.430+0000 7f96145ff700 1 -- start start 2023-07-06T21:43:03.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.431+0000 7f96145ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c0b83c0 0x7f960c14db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.431+0000 7f96145ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c1594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.431+0000 7f96145ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 0x7f960c159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.431+0000 7f96145ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c154290 con 0x7f960c0d3710 2023-07-06T21:43:03.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.431+0000 7f96145ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960c0ccaf0 con 0x7f960c0b83c0 2023-07-06T21:43:03.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.431+0000 7f96145ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960c153680 con 0x7f960c0cfc60 2023-07-06T21:43:03.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 0x7f960c159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 0x7f960c159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38696/0 (socket says 172.21.15.84:38696) 2023-07-06T21:43:03.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 -- 172.21.15.84:0/2316906807 learned_addr learned my addr 172.21.15.84:0/2316906807 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:03.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f961239b700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c0b83c0 0x7f960c14db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9611b9a700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c1594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 -- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 msgr2=0x7f960c1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c1594c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 -- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c0b83c0 msgr2=0x7f960c14db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c0b83c0 0x7f960c14db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 -- 172.21.15.84:0/2316906807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9608009550 con 0x7f960c0d3710 2023-07-06T21:43:03.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.432+0000 7f9612b9c700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 0x7f960c159a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f960400cdb0 tx=0x7f960400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.433+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400ae70 con 0x7f960c0d3710 2023-07-06T21:43:03.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.433+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960400f480 con 0x7f960c0d3710 2023-07-06T21:43:03.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.433+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960c153960 con 0x7f960c0d3710 2023-07-06T21:43:03.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.433+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960400f850 con 0x7f960c0d3710 2023-07-06T21:43:03.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.433+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960c153f20 con 0x7f960c0d3710 2023-07-06T21:43:03.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.434+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f960c1540e0 con 0x7f960c0d3710 2023-07-06T21:43:03.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.435+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f960400b040 con 0x7f960c0d3710 2023-07-06T21:43:03.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.436+0000 7f9602ffd700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95f006ace0 0x7f95f006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.436+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9604089840 con 0x7f960c0d3710 2023-07-06T21:43:03.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.436+0000 7f961239b700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95f006ace0 0x7f95f006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.437+0000 7f961239b700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95f006ace0 0x7f95f006d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9608014040 tx=0x7f9608008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.440+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9604054bf0 con 0x7f960c0d3710 2023-07-06T21:43:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.634+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f960c0d0710 con 0x7f95f006ace0 2023-07-06T21:43:03.636 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.634+0000 7f9602ffd700 1 -- 172.21.15.84:0/2316906807 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f960800bca0 con 0x7f95f006ace0 2023-07-06T21:43:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95f006ace0 msgr2=0x7f95f006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95f006ace0 0x7f95f006d190 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9608014040 tx=0x7f9608008030 comp rx=0 tx=0).stop 2023-07-06T21:43:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 msgr2=0x7f960c159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 0x7f960c159a00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f960400cdb0 tx=0x7f960400a560 comp rx=0 tx=0).stop 2023-07-06T21:43:03.639 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 shutdown_connections 2023-07-06T21:43:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f95f006ace0 0x7f95f006d190 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f960c0d3710 0x7f960c159a00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.636+0000 7f96145ff700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960c0b83c0 0x7f960c14db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.640 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.637+0000 7f96145ff700 1 --2- 172.21.15.84:0/2316906807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960c0cfc60 0x7f960c1594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.637+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 >> 172.21.15.84:0/2316906807 conn(0x7f960c06b350 msgr2=0x7f960c15a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:03.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.637+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 shutdown_connections 2023-07-06T21:43:03.641 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.637+0000 7f96145ff700 1 -- 172.21.15.84:0/2316906807 wait complete. 2023-07-06T21:43:03.642 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 3 to deep-scrub 2023-07-06T21:43:03.657 INFO:tasks.ceph:Scrubbing osd.4 2023-07-06T21:43:03.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:03.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.851+0000 7feb97fff700 1 Processor -- start 2023-07-06T21:43:03.853 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.851+0000 7feb97fff700 1 -- start start 2023-07-06T21:43:03.854 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.852+0000 7feb97fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7feb9015c830 0x7feb900af600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.852+0000 7feb97fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb900b09a0 0x7feb900c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.852+0000 7feb97fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb9006ae10 con 0x7feb900cf430 2023-07-06T21:43:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.852+0000 7feb97fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb900ce3d0 con 0x7feb900b09a0 2023-07-06T21:43:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.852+0000 7feb97fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb900ce670 con 0x7feb9015c830 2023-07-06T21:43:03.855 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb900b09a0 0x7feb900c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7feb900b09a0 0x7feb900c9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38710/0 (socket says 172.21.15.84:38710) 2023-07-06T21:43:03.856 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 -- 172.21.15.84:0/1413261352 learned_addr learned my addr 172.21.15.84:0/1413261352 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 -- 172.21.15.84:0/1413261352 >> v2:172.21.15.107:3300/0 conn(0x7feb9015c830 msgr2=0x7feb900af600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:03.857 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 --2- 172.21.15.84:0/1413261352 >> v2:172.21.15.107:3300/0 conn(0x7feb9015c830 0x7feb900af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 -- 172.21.15.84:0/1413261352 >> v1:172.21.15.107:6789/0 conn(0x7feb900cf430 legacy=0x7feb900b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 -- 172.21.15.84:0/1413261352 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb900ce950 con 0x7feb900b09a0 2023-07-06T21:43:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb9659c700 1 --2- 172.21.15.84:0/1413261352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900c9ef0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7feb9006ff20 tx=0x7feb8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c684cc31570a6fe server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb94d99700 1 -- 172.21.15.84:0/1413261352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800e070 con 0x7feb900b09a0 2023-07-06T21:43:03.858 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.853+0000 7feb94d99700 1 -- 172.21.15.84:0/1413261352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb88003a20 con 0x7feb900b09a0 2023-07-06T21:43:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb94d99700 1 -- 172.21.15.84:0/1413261352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88003dd0 con 0x7feb900b09a0 2023-07-06T21:43:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 -- 172.21.15.84:0/1413261352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 msgr2=0x7feb900c9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 --2- 172.21.15.84:0/1413261352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900c9ef0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7feb9006ff20 tx=0x7feb8800b330 comp rx=0 tx=0).stop 2023-07-06T21:43:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 -- 172.21.15.84:0/1413261352 shutdown_connections 2023-07-06T21:43:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 --2- 172.21.15.84:0/1413261352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900c9ef0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 --2- 172.21.15.84:0/1413261352 >> v2:172.21.15.107:3300/0 conn(0x7feb9015c830 0x7feb900af600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 -- 172.21.15.84:0/1413261352 >> 172.21.15.84:0/1413261352 conn(0x7feb9006b8c0 msgr2=0x7feb900cf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 -- 172.21.15.84:0/1413261352 shutdown_connections 2023-07-06T21:43:03.860 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.854+0000 7feb97fff700 1 -- 172.21.15.84:0/1413261352 wait complete. 2023-07-06T21:43:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.856+0000 7feb97fff700 1 Processor -- start 2023-07-06T21:43:03.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.856+0000 7feb97fff700 1 -- start start 2023-07-06T21:43:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb97fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb97fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb900cf430 0x7feb90156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb97fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb9015c830 0x7feb90156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb97fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c4140 con 0x7feb900b09a0 2023-07-06T21:43:03.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb97fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb90157390 con 0x7feb900cf430 2023-07-06T21:43:03.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb97fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb900c3500 con 0x7feb9015c830 2023-07-06T21:43:03.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb95d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb95d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38724/0 (socket says 172.21.15.84:38724) 2023-07-06T21:43:03.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb95d9b700 1 -- 172.21.15.84:0/1530162893 learned_addr learned my addr 172.21.15.84:0/1530162893 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:03.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb9559a700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb900cf430 0x7feb90156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.857+0000 7feb9659c700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb9015c830 0x7feb90156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb95d9b700 1 -- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb9015c830 msgr2=0x7feb90156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb95d9b700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb9015c830 0x7feb90156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb95d9b700 1 -- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb900cf430 msgr2=0x7feb90156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb95d9b700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb900cf430 0x7feb90156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb95d9b700 1 -- 172.21.15.84:0/1530162893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb88009b30 con 0x7feb900b09a0 2023-07-06T21:43:03.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb9559a700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb900cf430 0x7feb90156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-06T21:43:03.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb95d9b700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900b2940 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00b4a0 tx=0x7feb8c00b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c011830 con 0x7feb900b09a0 2023-07-06T21:43:03.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c011990 con 0x7feb900b09a0 2023-07-06T21:43:03.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb900c37a0 con 0x7feb900b09a0 2023-07-06T21:43:03.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.858+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c011d40 con 0x7feb900b09a0 2023-07-06T21:43:03.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.859+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb900c3d60 con 0x7feb900b09a0 2023-07-06T21:43:03.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.859+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb900cef10 con 0x7feb900b09a0 2023-07-06T21:43:03.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.860+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7feb8c01a400 con 0x7feb900b09a0 2023-07-06T21:43:03.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.861+0000 7feb867fc700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7406ace0 0x7feb7406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.861+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7feb8c089120 con 0x7feb900b09a0 2023-07-06T21:43:03.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.861+0000 7feb9559a700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7406ace0 0x7feb7406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.862+0000 7feb867fc700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7feb74071150 0x7feb74073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:03.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.862+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 --> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb74073c30 con 0x7feb74071150 2023-07-06T21:43:03.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.862+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb8c08b700 con 0x7feb900b09a0 2023-07-06T21:43:03.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.862+0000 7feb9659c700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7feb74071150 0x7feb74073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:03.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.862+0000 7feb9559a700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7406ace0 0x7feb7406d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb9009d7f0 tx=0x7feb80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.863+0000 7feb9659c700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7feb74071150 0x7feb74073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:03.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.864+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== osd.4 v2:172.21.15.107:6800/1667769091 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7feb74073c30 con 0x7feb74071150 2023-07-06T21:43:03.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.878+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 --> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feb900cec30 con 0x7feb74071150 2023-07-06T21:43:03.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.879+0000 7feb867fc700 1 -- 172.21.15.84:0/1530162893 <== osd.4 v2:172.21.15.107:6800/1667769091 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feb88003e30 con 0x7feb74071150 2023-07-06T21:43:03.881 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:03.882 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:03.882 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:03.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.879+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7feb74071150 msgr2=0x7feb74073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.879+0000 7feb97fff700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.107:6800/1667769091,v1:172.21.15.107:6801/1667769091] conn(0x7feb74071150 0x7feb74073560 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7406ace0 msgr2=0x7feb7406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.883 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7feb7406ace0 0x7feb7406d190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb9009d7f0 tx=0x7feb80009040 comp rx=0 tx=0).stop 2023-07-06T21:43:03.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 msgr2=0x7feb900b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:03.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 --2- 172.21.15.84:0/1530162893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb900b09a0 0x7feb900b2940 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00b4a0 tx=0x7feb8c00b7b0 comp rx=0 tx=0).stop 2023-07-06T21:43:03.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb9659c700 1 -- 172.21.15.84:0/1530162893 reap_dead start 2023-07-06T21:43:03.884 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 shutdown_connections 2023-07-06T21:43:03.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 >> 172.21.15.84:0/1530162893 conn(0x7feb9006b8c0 msgr2=0x7feb900cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:03.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 shutdown_connections 2023-07-06T21:43:03.885 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:03.880+0000 7feb97fff700 1 -- 172.21.15.84:0/1530162893 wait complete. 2023-07-06T21:43:03.898 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-07-06T21:43:04.093 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.091+0000 7fbfc6bff700 1 Processor -- start 2023-07-06T21:43:04.094 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.091+0000 7fbfc6bff700 1 -- start start 2023-07-06T21:43:04.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.092+0000 7fbfc6bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 0x7fbfc00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.092+0000 7fbfc6bff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc00d0a80 con 0x7fbfc00d3710 2023-07-06T21:43:04.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfc6bff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc00b8790 con 0x7fbfc00cfc60 2023-07-06T21:43:04.095 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfc6bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b4e70 con 0x7fbfc00b83c0 2023-07-06T21:43:04.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbffff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fbfc00d3710 0x7fbfc00cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34056/0 (socket says 172.21.15.84:34056) 2023-07-06T21:43:04.096 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbffff700 1 -- 172.21.15.84:0/3248536569 learned_addr learned my addr 172.21.15.84:0/3248536569 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:04.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfc499b700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 0x7fbfc00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.098 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061327583 0 0) 0x7fbfc00d0a80 con 0x7fbfc00d3710 2023-07-06T21:43:04.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfb00035b0 con 0x7fbfc00d3710 2023-07-06T21:43:04.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572691943 0 0) 0x7fbfb00035b0 con 0x7fbfc00d3710 2023-07-06T21:43:04.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 msgr2=0x7fbfc00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 0x7fbfc00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.099 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 >> v1:172.21.15.107:6789/0 conn(0x7fbfc00cfc60 legacy=0x7fbfc00d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc00b5150 con 0x7fbfc00d3710 2023-07-06T21:43:04.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.093+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7fbfac001670 con 0x7fbfc00d3710 2023-07-06T21:43:04.100 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 >> v1:172.21.15.84:6789/0 conn(0x7fbfc00d3710 legacy=0x7fbfc00cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfb0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb00043d0 0x7fbfb0004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 0x7fbfb0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.101 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc00b4e70 con 0x7fbfb0003a30 2023-07-06T21:43:04.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb000bac0 con 0x7fbfb00043d0 2023-07-06T21:43:04.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb000bc50 con 0x7fbfc00b83c0 2023-07-06T21:43:04.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc499b700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfb0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfbffff700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb00043d0 0x7fbfb0004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.102 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc519c700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 0x7fbfb0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc499b700 1 -- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 msgr2=0x7fbfb0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc499b700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00b83c0 0x7fbfb0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc499b700 1 -- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb00043d0 msgr2=0x7fbfb0004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc499b700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb00043d0 0x7fbfb0004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.094+0000 7fbfc499b700 1 -- 172.21.15.84:0/3248536569 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb00035b0 con 0x7fbfb0003a30 2023-07-06T21:43:04.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc499b700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfb0003e00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009b90 tx=0x7fbfb0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16872244a9998fb5 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400e6c0 con 0x7fbfb0003a30 2023-07-06T21:43:04.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb400e820 con 0x7fbfb0003a30 2023-07-06T21:43:04.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfbf7fe700 1 -- 172.21.15.84:0/3248536569 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400eb90 con 0x7fbfb0003a30 2023-07-06T21:43:04.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc6bff700 1 -- 172.21.15.84:0/3248536569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 msgr2=0x7fbfb0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/3248536569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfb0003e00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbfb4009b90 tx=0x7fbfb0005880 comp rx=0 tx=0).stop 2023-07-06T21:43:04.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc519c700 1 -- 172.21.15.84:0/3248536569 reap_dead start 2023-07-06T21:43:04.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc6bff700 1 -- 172.21.15.84:0/3248536569 shutdown_connections 2023-07-06T21:43:04.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc6bff700 1 -- 172.21.15.84:0/3248536569 >> 172.21.15.84:0/3248536569 conn(0x7fbfc006b350 msgr2=0x7fbfc006b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:04.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc6bff700 1 -- 172.21.15.84:0/3248536569 shutdown_connections 2023-07-06T21:43:04.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.095+0000 7fbfc6bff700 1 -- 172.21.15.84:0/3248536569 wait complete. 2023-07-06T21:43:04.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.097+0000 7fbfc6bff700 1 Processor -- start 2023-07-06T21:43:04.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.097+0000 7fbfc6bff700 1 -- start start 2023-07-06T21:43:04.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc6bff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfc00d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc6bff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00c8e10 0x7fbfc01f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc6bff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfc00c9280 0x7fbfc01f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc6bff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01f21b0 con 0x7fbfb0003a30 2023-07-06T21:43:04.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc6bff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01f2490 con 0x7fbfc00c9280 2023-07-06T21:43:04.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc6bff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc01f2770 con 0x7fbfc00c8e10 2023-07-06T21:43:04.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc499b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfc00d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc499b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfc00d2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38756/0 (socket says 172.21.15.84:38756) 2023-07-06T21:43:04.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.098+0000 7fbfc499b700 1 -- 172.21.15.84:0/2232943445 learned_addr learned my addr 172.21.15.84:0/2232943445 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:04.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfbffff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00c8e10 0x7fbfc01f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc519c700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfc00c9280 0x7fbfc01f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc499b700 1 -- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00c8e10 msgr2=0x7fbfc01f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc499b700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00c8e10 0x7fbfc01f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc499b700 1 -- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfc00c9280 msgr2=0x7fbfc01f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc499b700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfc00c9280 0x7fbfc01f57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc499b700 1 -- 172.21.15.84:0/2232943445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc01f2a50 con 0x7fbfb0003a30 2023-07-06T21:43:04.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc519c700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfc00c9280 0x7fbfc01f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:04.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc499b700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfc00d2c40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00057f0 tx=0x7fbfb40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400e6c0 con 0x7fbfb0003a30 2023-07-06T21:43:04.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb4003ad0 con 0x7fbfb0003a30 2023-07-06T21:43:04.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc00d3180 con 0x7fbfb0003a30 2023-07-06T21:43:04.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.099+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4003e80 con 0x7fbfb0003a30 2023-07-06T21:43:04.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.100+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc01f5ef0 con 0x7fbfb0003a30 2023-07-06T21:43:04.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.100+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc01f3090 con 0x7fbfb0003a30 2023-07-06T21:43:04.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.101+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fbfb4013470 con 0x7fbfb0003a30 2023-07-06T21:43:04.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.102+0000 7fbfbd7fa700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfa406ace0 0x7fbfa406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.103+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fbfb408aa80 con 0x7fbfb0003a30 2023-07-06T21:43:04.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.103+0000 7fbfbffff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfa406ace0 0x7fbfa406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.103+0000 7fbfbffff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfa406ace0 0x7fbfa406d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00b6a10 tx=0x7fbfac001000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.107+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fbfb4055f60 con 0x7fbfb0003a30 2023-07-06T21:43:04.305 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.303+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfc01f2600 con 0x7fbfa406ace0 2023-07-06T21:43:04.308 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.303+0000 7fbfbd7fa700 1 -- 172.21.15.84:0/2232943445 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbfc01f2600 con 0x7fbfa406ace0 2023-07-06T21:43:04.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfa406ace0 msgr2=0x7fbfa406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfa406ace0 0x7fbfa406d190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbfc00b6a10 tx=0x7fbfac001000 comp rx=0 tx=0).stop 2023-07-06T21:43:04.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 msgr2=0x7fbfc00d2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.315 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfc00d2c40 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fbfb00057f0 tx=0x7fbfb40047c0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 shutdown_connections 2023-07-06T21:43:04.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fbfa406ace0 0x7fbfa406d190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfc00c9280 0x7fbfc01f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.316 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbfb0003a30 0x7fbfc00d2c40 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 --2- 172.21.15.84:0/2232943445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfc00c8e10 0x7fbfc01f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 >> 172.21.15.84:0/2232943445 conn(0x7fbfc006b350 msgr2=0x7fbfc00b7080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:04.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 shutdown_connections 2023-07-06T21:43:04.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.305+0000 7fbfc6bff700 1 -- 172.21.15.84:0/2232943445 wait complete. 2023-07-06T21:43:04.317 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 4 to deep-scrub 2023-07-06T21:43:04.325 INFO:tasks.ceph:Scrubbing osd.5 2023-07-06T21:43:04.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:04.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.521+0000 7fab94dff700 1 Processor -- start 2023-07-06T21:43:04.523 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.521+0000 7fab94dff700 1 -- start start 2023-07-06T21:43:04.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab94dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab900d0ea0 0x7fab900d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.524 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab94dff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fab9015fee0 0x7fab90160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.525 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab94dff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab900ceef0 con 0x7fab90160ac0 2023-07-06T21:43:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab94dff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab900cf450 con 0x7fab9015fee0 2023-07-06T21:43:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab94dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab900cf120 con 0x7fab900d0ea0 2023-07-06T21:43:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab8ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fab9015fee0 0x7fab90160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.526 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab8ed9d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fab9015fee0 0x7fab90160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38772/0 (socket says 172.21.15.84:38772) 2023-07-06T21:43:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.522+0000 7fab8ed9d700 1 -- 172.21.15.84:0/1240903500 learned_addr learned my addr 172.21.15.84:0/1240903500 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:04.528 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8e59c700 1 --2- 172.21.15.84:0/1240903500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab900d0ea0 0x7fab900d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8ed9d700 1 -- 172.21.15.84:0/1240903500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab900d0ea0 msgr2=0x7fab900d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8ed9d700 1 --2- 172.21.15.84:0/1240903500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab900d0ea0 0x7fab900d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8ed9d700 1 -- 172.21.15.84:0/1240903500 >> v1:172.21.15.107:6789/0 conn(0x7fab90160ac0 legacy=0x7fab900cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8ed9d700 1 -- 172.21.15.84:0/1240903500 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab90160880 con 0x7fab9015fee0 2023-07-06T21:43:04.529 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8ed9d700 1 --2- 172.21.15.84:0/1240903500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab9015fee0 0x7fab90160340 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fab88009e50 tx=0x7fab8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902f3f68db1d6ecb server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8d59a700 1 -- 172.21.15.84:0/1240903500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800e070 con 0x7fab9015fee0 2023-07-06T21:43:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8d59a700 1 -- 172.21.15.84:0/1240903500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8800b9c0 con 0x7fab9015fee0 2023-07-06T21:43:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.523+0000 7fab8d59a700 1 -- 172.21.15.84:0/1240903500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8800bd70 con 0x7fab9015fee0 2023-07-06T21:43:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 -- 172.21.15.84:0/1240903500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab9015fee0 msgr2=0x7fab90160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 --2- 172.21.15.84:0/1240903500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab9015fee0 0x7fab90160340 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fab88009e50 tx=0x7fab8800b330 comp rx=0 tx=0).stop 2023-07-06T21:43:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 -- 172.21.15.84:0/1240903500 shutdown_connections 2023-07-06T21:43:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 --2- 172.21.15.84:0/1240903500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab9015fee0 0x7fab90160340 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 --2- 172.21.15.84:0/1240903500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab900d0ea0 0x7fab900d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 -- 172.21.15.84:0/1240903500 >> 172.21.15.84:0/1240903500 conn(0x7fab900715c0 msgr2=0x7fab90071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:04.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 -- 172.21.15.84:0/1240903500 shutdown_connections 2023-07-06T21:43:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.524+0000 7fab94dff700 1 -- 172.21.15.84:0/1240903500 wait complete. 2023-07-06T21:43:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.526+0000 7fab94dff700 1 Processor -- start 2023-07-06T21:43:04.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.526+0000 7fab94dff700 1 -- start start 2023-07-06T21:43:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab94dff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab9015fee0 0x7fab900cc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.534 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab94dff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab90160ac0 0x7fab900c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab94dff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab900cb4d0 0x7fab901521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab94dff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab90157860 con 0x7fab90160ac0 2023-07-06T21:43:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab94dff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900c9370 con 0x7fab900cb4d0 2023-07-06T21:43:04.535 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab94dff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab90156f70 con 0x7fab9015fee0 2023-07-06T21:43:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab8dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab90160ac0 0x7fab900c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab8dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab90160ac0 0x7fab900c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38788/0 (socket says 172.21.15.84:38788) 2023-07-06T21:43:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab8dd9b700 1 -- 172.21.15.84:0/3047781322 learned_addr learned my addr 172.21.15.84:0/3047781322 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:04.536 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab8ed9d700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab900cb4d0 0x7fab901521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.527+0000 7fab8e59c700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab9015fee0 0x7fab900cc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab8dd9b700 1 -- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab9015fee0 msgr2=0x7fab900cc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab8dd9b700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab9015fee0 0x7fab900cc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab8dd9b700 1 -- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab900cb4d0 msgr2=0x7fab901521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.537 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab8dd9b700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab900cb4d0 0x7fab901521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab8dd9b700 1 -- 172.21.15.84:0/3047781322 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88009b30 con 0x7fab90160ac0 2023-07-06T21:43:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab8dd9b700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab90160ac0 0x7fab900c67d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c4b0 tx=0x7fab8400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840052d0 con 0x7fab90160ac0 2023-07-06T21:43:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab84005430 con 0x7fab90160ac0 2023-07-06T21:43:04.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab900cca80 con 0x7fab90160ac0 2023-07-06T21:43:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab840038c0 con 0x7fab90160ac0 2023-07-06T21:43:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.528+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab901574f0 con 0x7fab90160ac0 2023-07-06T21:43:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.529+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab900cd700 con 0x7fab90160ac0 2023-07-06T21:43:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.530+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fab84003a20 con 0x7fab90160ac0 2023-07-06T21:43:04.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.531+0000 7fab7effd700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fab7006ace0 0x7fab7006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.531+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fab84088530 con 0x7fab90160ac0 2023-07-06T21:43:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.531+0000 7fab8e59c700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fab7006ace0 0x7fab7006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.532+0000 7fab7effd700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fab70071150 0x7fab70073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.532+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 --> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab70073c30 con 0x7fab70071150 2023-07-06T21:43:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.532+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab8408ab10 con 0x7fab90160ac0 2023-07-06T21:43:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.532+0000 7fab8e59c700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fab7006ace0 0x7fab7006d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fab80006fd0 tx=0x7fab80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.532+0000 7fab8ed9d700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fab70071150 0x7fab70073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.533+0000 7fab8ed9d700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fab70071150 0x7fab70073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.533+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== osd.5 v2:172.21.15.107:6808/3858042795 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fab70073c30 con 0x7fab70071150 2023-07-06T21:43:04.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.548+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 --> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fab900cd420 con 0x7fab70071150 2023-07-06T21:43:04.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab7effd700 1 -- 172.21.15.84:0/3047781322 <== osd.5 v2:172.21.15.107:6808/3858042795 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fab8800bd50 con 0x7fab70071150 2023-07-06T21:43:04.551 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:04.551 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:04.551 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:04.552 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fab70071150 msgr2=0x7fab70073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.107:6808/3858042795,v1:172.21.15.107:6809/3858042795] conn(0x7fab70071150 0x7fab70073560 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fab7006ace0 msgr2=0x7fab7006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fab7006ace0 0x7fab7006d190 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fab80006fd0 tx=0x7fab80008040 comp rx=0 tx=0).stop 2023-07-06T21:43:04.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab90160ac0 msgr2=0x7fab900c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.553 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 --2- 172.21.15.84:0/3047781322 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fab90160ac0 0x7fab900c67d0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fab8400c4b0 tx=0x7fab8400c7c0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab8ed9d700 1 -- 172.21.15.84:0/3047781322 reap_dead start 2023-07-06T21:43:04.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 shutdown_connections 2023-07-06T21:43:04.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 >> 172.21.15.84:0/3047781322 conn(0x7fab900715c0 msgr2=0x7fab900d27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:04.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 shutdown_connections 2023-07-06T21:43:04.554 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.549+0000 7fab94dff700 1 -- 172.21.15.84:0/3047781322 wait complete. 2023-07-06T21:43:04.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-07-06T21:43:04.762 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.760+0000 7f71539ff700 1 Processor -- start 2023-07-06T21:43:04.763 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.760+0000 7f71539ff700 1 -- start start 2023-07-06T21:43:04.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f71539ff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f714c0d1580 0x7f714c166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f71539ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714c0d0990 con 0x7f714c0d3580 2023-07-06T21:43:04.764 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f71539ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714c0cd9d0 con 0x7f714c0b8530 2023-07-06T21:43:04.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f71539ff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714c0d3af0 con 0x7f714c0d1580 2023-07-06T21:43:04.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f7150f9a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f714c0d3580 0x7f714c0d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34072/0 (socket says 172.21.15.84:34072) 2023-07-06T21:43:04.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f7150f9a700 1 -- 172.21.15.84:0/4195417752 learned_addr learned my addr 172.21.15.84:0/4195417752 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:04.765 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f7151f9c700 1 --2- 172.21.15.84:0/4195417752 >> v2:172.21.15.107:3300/0 conn(0x7f714c0d1580 0x7f714c166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818554779 0 0) 0x7f714c0d0990 con 0x7f714c0d3580 2023-07-06T21:43:04.766 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.762+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71380035b0 con 0x7f714c0d3580 2023-07-06T21:43:04.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263681379 0 0) 0x7f714c0cd9d0 con 0x7f714c0b8530 2023-07-06T21:43:04.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f714c0d0990 con 0x7f714c0b8530 2023-07-06T21:43:04.768 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3815637938 0 0) 0x7f71380035b0 con 0x7f714c0d3580 2023-07-06T21:43:04.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 >> v2:172.21.15.107:3300/0 conn(0x7f714c0d1580 msgr2=0x7f714c166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 --2- 172.21.15.84:0/4195417752 >> v2:172.21.15.107:3300/0 conn(0x7f714c0d1580 0x7f714c166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 >> v1:172.21.15.107:6789/0 conn(0x7f714c0b8530 legacy=0x7f714c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c0cd410 con 0x7f714c0d3580 2023-07-06T21:43:04.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f713c001670 con 0x7f714c0d3580 2023-07-06T21:43:04.769 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 >> v1:172.21.15.84:6789/0 conn(0x7f714c0d3580 legacy=0x7f714c0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138003be0 0x7f7138003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138004850 0x7f7138004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7138005200 0x7f713800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.770 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c0d3af0 con 0x7f7138005200 2023-07-06T21:43:04.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713800c0e0 con 0x7f7138003be0 2023-07-06T21:43:04.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f713800c270 con 0x7f7138004850 2023-07-06T21:43:04.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7151f9c700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7138005200 0x7f713800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.771 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7150f9a700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138004850 0x7f7138004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f715179b700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138003be0 0x7f7138003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7151f9c700 1 -- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138004850 msgr2=0x7f7138004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7151f9c700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138004850 0x7f7138004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7151f9c700 1 -- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138003be0 msgr2=0x7f7138003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:04.772 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7151f9c700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7138003be0 0x7f7138003fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.763+0000 7f7151f9c700 1 -- 172.21.15.84:0/4195417752 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71380035b0 con 0x7f7138005200 2023-07-06T21:43:04.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f7151f9c700 1 -- 172.21.15.84:0/4195417752 reap_dead start 2023-07-06T21:43:04.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f7151f9c700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7138005200 0x7f713800ba10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7144009f90 tx=0x7f7138005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d67b72f35aaa1 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.773 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144003a20 con 0x7f7138005200 2023-07-06T21:43:04.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714400f040 con 0x7f7138005200 2023-07-06T21:43:04.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f7143fff700 1 -- 172.21.15.84:0/4195417752 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144003dc0 con 0x7f7138005200 2023-07-06T21:43:04.774 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f71539ff700 1 -- 172.21.15.84:0/4195417752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7138005200 msgr2=0x7f713800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f71539ff700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7138005200 0x7f713800ba10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7144009f90 tx=0x7f7138005880 comp rx=0 tx=0).stop 2023-07-06T21:43:04.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f71539ff700 1 -- 172.21.15.84:0/4195417752 shutdown_connections 2023-07-06T21:43:04.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f71539ff700 1 --2- 172.21.15.84:0/4195417752 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7138005200 0x7f713800ba10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.775 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.764+0000 7f71539ff700 1 -- 172.21.15.84:0/4195417752 >> 172.21.15.84:0/4195417752 conn(0x7f714c0715c0 msgr2=0x7f714c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:04.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.765+0000 7f71539ff700 1 -- 172.21.15.84:0/4195417752 shutdown_connections 2023-07-06T21:43:04.776 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.765+0000 7f71539ff700 1 -- 172.21.15.84:0/4195417752 wait complete. 2023-07-06T21:43:04.778 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.767+0000 7f71539ff700 1 Processor -- start 2023-07-06T21:43:04.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.767+0000 7f71539ff700 1 -- start start 2023-07-06T21:43:04.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.768+0000 7f71539ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138005200 0x7f714c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.768+0000 7f71539ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 0x7f714c0b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.779 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.768+0000 7f71539ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c0cc1b0 0x7f714c0cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.768+0000 7f71539ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c1f2d70 con 0x7f714c0b2b60 2023-07-06T21:43:04.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.768+0000 7f71539ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c0ccb60 con 0x7f714c0cc1b0 2023-07-06T21:43:04.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.768+0000 7f71539ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c1f1d50 con 0x7f7138005200 2023-07-06T21:43:04.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 0x7f714c0b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.780 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 0x7f714c0b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38812/0 (socket says 172.21.15.84:38812) 2023-07-06T21:43:04.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 -- 172.21.15.84:0/4243225923 learned_addr learned my addr 172.21.15.84:0/4243225923 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:04.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f715179b700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138005200 0x7f714c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7151f9c700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c0cc1b0 0x7f714c0cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 -- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138005200 msgr2=0x7f714c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.781 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138005200 0x7f714c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 -- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c0cc1b0 msgr2=0x7f714c0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c0cc1b0 0x7f714c0cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 -- 172.21.15.84:0/4243225923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c1f1fb0 con 0x7f714c0b2b60 2023-07-06T21:43:04.782 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7150f9a700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 0x7f714c0b2fd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f713c0072f0 tx=0x7f713c007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f640 con 0x7f714c0b2b60 2023-07-06T21:43:04.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.769+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00f7a0 con 0x7f714c0b2b60 2023-07-06T21:43:04.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.770+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c1f2290 con 0x7f714c0b2b60 2023-07-06T21:43:04.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.770+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00fb50 con 0x7f714c0b2b60 2023-07-06T21:43:04.783 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.770+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714c1f2850 con 0x7f714c0b2b60 2023-07-06T21:43:04.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.771+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c1f2a10 con 0x7f714c0b2b60 2023-07-06T21:43:04.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.772+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f713c00fcb0 con 0x7f714c0b2b60 2023-07-06T21:43:04.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.772+0000 7f7141ffb700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f713006ace0 0x7f713006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:04.784 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.773+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f713c088800 con 0x7f714c0b2b60 2023-07-06T21:43:04.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.773+0000 7f715179b700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f713006ace0 0x7f713006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:04.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.773+0000 7f715179b700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f713006ace0 0x7f713006d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7148000ef0 tx=0x7f7148009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:04.785 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.777+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f713c053b30 con 0x7f714c0b2b60 2023-07-06T21:43:04.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.973+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f714c0b4fc0 con 0x7f713006ace0 2023-07-06T21:43:04.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.973+0000 7f7141ffb700 1 -- 172.21.15.84:0/4243225923 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7148005270 con 0x7f713006ace0 2023-07-06T21:43:04.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f713006ace0 msgr2=0x7f713006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f713006ace0 0x7f713006d190 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7148000ef0 tx=0x7f7148009040 comp rx=0 tx=0).stop 2023-07-06T21:43:04.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 msgr2=0x7f714c0b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:04.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 0x7f714c0b2fd0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f713c0072f0 tx=0x7f713c007600 comp rx=0 tx=0).stop 2023-07-06T21:43:04.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 shutdown_connections 2023-07-06T21:43:04.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f713006ace0 0x7f713006d190 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c0cc1b0 0x7f714c0cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7138005200 0x7f714c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 --2- 172.21.15.84:0/4243225923 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f714c0b2b60 0x7f714c0b2fd0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:04.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 >> 172.21.15.84:0/4243225923 conn(0x7f714c0715c0 msgr2=0x7f714c0733a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:04.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 shutdown_connections 2023-07-06T21:43:04.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:04.976+0000 7f71539ff700 1 -- 172.21.15.84:0/4243225923 wait complete. 2023-07-06T21:43:04.981 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 5 to deep-scrub 2023-07-06T21:43:04.996 INFO:tasks.ceph:Scrubbing osd.6 2023-07-06T21:43:04.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:05.194 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.192+0000 7f7cbbdff700 1 Processor -- start 2023-07-06T21:43:05.195 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.192+0000 7f7cbbdff700 1 -- start start 2023-07-06T21:43:05.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.193+0000 7f7cbbdff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7cb40d0ea0 0x7f7cb40d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.193+0000 7f7cbbdff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7cb4160ac0 0x7f7cb40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.197 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.193+0000 7f7cbbdff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb40ceef0 con 0x7f7cb40cfe20 2023-07-06T21:43:05.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.193+0000 7f7cbbdff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40cf450 con 0x7f7cb4160ac0 2023-07-06T21:43:05.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.193+0000 7f7cbbdff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40cf120 con 0x7f7cb40d0ea0 2023-07-06T21:43:05.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7cb4160ac0 0x7f7cb40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.198 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7cb4160ac0 0x7f7cb40cf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38820/0 (socket says 172.21.15.84:38820) 2023-07-06T21:43:05.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 -- 172.21.15.84:0/630171553 learned_addr learned my addr 172.21.15.84:0/630171553 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:05.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb9b9b700 1 --2- 172.21.15.84:0/630171553 >> v2:172.21.15.107:3300/0 conn(0x7f7cb40d0ea0 0x7f7cb40d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 -- 172.21.15.84:0/630171553 >> v2:172.21.15.107:3300/0 conn(0x7f7cb40d0ea0 msgr2=0x7f7cb40d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:05.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 --2- 172.21.15.84:0/630171553 >> v2:172.21.15.107:3300/0 conn(0x7f7cb40d0ea0 0x7f7cb40d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 -- 172.21.15.84:0/630171553 >> v1:172.21.15.107:6789/0 conn(0x7f7cb40cfe20 legacy=0x7f7cb415fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.199 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 -- 172.21.15.84:0/630171553 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40cd420 con 0x7f7cb4160ac0 2023-07-06T21:43:05.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.194+0000 7f7cb939a700 1 --2- 172.21.15.84:0/630171553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4160ac0 0x7f7cb40cf8e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40b4e10 tx=0x7f7ca4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f3cc3e05fd1a0a server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cb8b99700 1 -- 172.21.15.84:0/630171553 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400b870 con 0x7f7cb4160ac0 2023-07-06T21:43:05.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cb8b99700 1 -- 172.21.15.84:0/630171553 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca400b9d0 con 0x7f7cb4160ac0 2023-07-06T21:43:05.200 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cb8b99700 1 -- 172.21.15.84:0/630171553 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400bd60 con 0x7f7cb4160ac0 2023-07-06T21:43:05.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 -- 172.21.15.84:0/630171553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4160ac0 msgr2=0x7f7cb40cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.202 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 --2- 172.21.15.84:0/630171553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4160ac0 0x7f7cb40cf8e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7cb40b4e10 tx=0x7f7ca4009bd0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 -- 172.21.15.84:0/630171553 shutdown_connections 2023-07-06T21:43:05.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 --2- 172.21.15.84:0/630171553 >> v2:172.21.15.107:3300/0 conn(0x7f7cb40d0ea0 0x7f7cb40d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 --2- 172.21.15.84:0/630171553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4160ac0 0x7f7cb40cf8e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.203 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 -- 172.21.15.84:0/630171553 >> 172.21.15.84:0/630171553 conn(0x7f7cb40715c0 msgr2=0x7f7cb4071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:05.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 -- 172.21.15.84:0/630171553 shutdown_connections 2023-07-06T21:43:05.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.195+0000 7f7cbbdff700 1 -- 172.21.15.84:0/630171553 wait complete. 2023-07-06T21:43:05.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.197+0000 7f7cbbdff700 1 Processor -- start 2023-07-06T21:43:05.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.197+0000 7f7cbbdff700 1 -- start start 2023-07-06T21:43:05.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cbbdff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb40cfe20 0x7f7cb40b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.204 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cbbdff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb4160ac0 0x7f7cb41f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cbbdff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4146a00 0x7f7cb415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cbbdff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4153bb0 con 0x7f7cb4146a00 2023-07-06T21:43:05.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cbbdff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb41f6ba0 con 0x7f7cb40cfe20 2023-07-06T21:43:05.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cbbdff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40b7b60 con 0x7f7cb4160ac0 2023-07-06T21:43:05.205 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cba39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4146a00 0x7f7cb415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.198+0000 7f7cba39c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4146a00 0x7f7cb415e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38832/0 (socket says 172.21.15.84:38832) 2023-07-06T21:43:05.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 -- 172.21.15.84:0/3263832139 learned_addr learned my addr 172.21.15.84:0/3263832139 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:05.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cb939a700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb4160ac0 0x7f7cb41f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cb9b9b700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb40cfe20 0x7f7cb40b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.206 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 -- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb4160ac0 msgr2=0x7f7cb41f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cb4160ac0 0x7f7cb41f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 -- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb40cfe20 msgr2=0x7f7cb40b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cb40cfe20 0x7f7cb40b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 -- 172.21.15.84:0/3263832139 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4009550 con 0x7f7cb4146a00 2023-07-06T21:43:05.207 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cba39c700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4146a00 0x7f7cb415e730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00cbc0 tx=0x7f7cac00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac003c00 con 0x7f7cb4146a00 2023-07-06T21:43:05.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac003d60 con 0x7f7cb4146a00 2023-07-06T21:43:05.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40b7e40 con 0x7f7cb4146a00 2023-07-06T21:43:05.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac00aae0 con 0x7f7cb4146a00 2023-07-06T21:43:05.208 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.199+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40b8400 con 0x7f7cb4146a00 2023-07-06T21:43:05.209 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.200+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cb40cd9e0 con 0x7f7cb4146a00 2023-07-06T21:43:05.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.201+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7cac00ac40 con 0x7f7cb4146a00 2023-07-06T21:43:05.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.202+0000 7f7caa7fc700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7c9806ace0 0x7f7c9806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.202+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7cac089120 con 0x7f7cb4146a00 2023-07-06T21:43:05.210 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.202+0000 7f7cb9b9b700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7c9806ace0 0x7f7c9806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.203+0000 7f7caa7fc700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7f7c98071150 0x7f7c98073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.203+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 --> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c98073c30 con 0x7f7c98071150 2023-07-06T21:43:05.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.203+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cac08b700 con 0x7f7cb4146a00 2023-07-06T21:43:05.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.203+0000 7f7cb9b9b700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7c9806ace0 0x7f7c9806d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006fd0 tx=0x7f7cb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.211 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.203+0000 7f7cb939a700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7f7c98071150 0x7f7c98073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.204+0000 7f7cb939a700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7f7c98071150 0x7f7c98073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.212 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.205+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== osd.6 v2:172.21.15.107:6816/2301301903 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7c98073c30 con 0x7f7c98071150 2023-07-06T21:43:05.221 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.219+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 --> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7cb40cd700 con 0x7f7c98071150 2023-07-06T21:43:05.222 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.220+0000 7f7caa7fc700 1 -- 172.21.15.84:0/3263832139 <== osd.6 v2:172.21.15.107:6816/2301301903 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7ca4021040 con 0x7f7c98071150 2023-07-06T21:43:05.222 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:05.223 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:05.223 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:05.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7f7c98071150 msgr2=0x7f7c98073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.107:6816/2301301903,v1:172.21.15.107:6817/2301301903] conn(0x7f7c98071150 0x7f7c98073560 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.224 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7c9806ace0 msgr2=0x7f7c9806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7c9806ace0 0x7f7c9806d190 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0006fd0 tx=0x7f7cb0008040 comp rx=0 tx=0).stop 2023-07-06T21:43:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4146a00 msgr2=0x7f7cb415e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 --2- 172.21.15.84:0/3263832139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cb4146a00 0x7f7cb415e730 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7cac00cbc0 tx=0x7f7cac00ced0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cba39c700 1 -- 172.21.15.84:0/3263832139 reap_dead start 2023-07-06T21:43:05.225 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 shutdown_connections 2023-07-06T21:43:05.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 >> 172.21.15.84:0/3263832139 conn(0x7f7cb40715c0 msgr2=0x7f7cb4152490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:05.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 shutdown_connections 2023-07-06T21:43:05.226 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.221+0000 7f7cbbdff700 1 -- 172.21.15.84:0/3263832139 wait complete. 2023-07-06T21:43:05.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-07-06T21:43:05.433 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.431+0000 7f67727ff700 1 Processor -- start 2023-07-06T21:43:05.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.431+0000 7f67727ff700 1 -- start start 2023-07-06T21:43:05.434 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.432+0000 7f67727ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0cfc60 0x7f676c167010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.432+0000 7f67727ff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0cd910 con 0x7f676c0b83c0 2023-07-06T21:43:05.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.432+0000 7f67727ff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676c0d0a80 con 0x7f676c0d3710 2023-07-06T21:43:05.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.432+0000 7f67727ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c0b8790 con 0x7f676c0cfc60 2023-07-06T21:43:05.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f676bfff700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f676c0b83c0 0x7f676c0cd200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34084/0 (socket says 172.21.15.84:34084) 2023-07-06T21:43:05.435 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f676bfff700 1 -- 172.21.15.84:0/2305878727 learned_addr learned my addr 172.21.15.84:0/2305878727 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:05.440 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f6770d9c700 1 --2- 172.21.15.84:0/2305878727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0cfc60 0x7f676c167010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009166089 0 0) 0x7f675c001110 con 0x7f676c0b83c0 2023-07-06T21:43:05.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67580035b0 con 0x7f676c0b83c0 2023-07-06T21:43:05.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759388157 0 0) 0x7f676c0d0a80 con 0x7f676c0d3710 2023-07-06T21:43:05.441 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f675c001110 con 0x7f676c0d3710 2023-07-06T21:43:05.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f6770d9c700 1 -- 172.21.15.84:0/2305878727 >> v1:172.21.15.107:6789/0 conn(0x7f676c0d3710 legacy=0x7f676c0cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f6770d9c700 1 -- 172.21.15.84:0/2305878727 >> v1:172.21.15.84:6789/0 conn(0x7f676c0b83c0 legacy=0x7f676c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.433+0000 7f6770d9c700 1 -- 172.21.15.84:0/2305878727 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0b4e70 con 0x7f676c0cfc60 2023-07-06T21:43:05.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.434+0000 7f6770d9c700 1 --2- 172.21.15.84:0/2305878727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0cfc60 0x7f676c167010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6764009dc0 tx=0x7f676400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b52d2f167e444a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.442 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.434+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676400e070 con 0x7f676c0cfc60 2023-07-06T21:43:05.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.434+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676400bb20 con 0x7f676c0cfc60 2023-07-06T21:43:05.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.434+0000 7f676b7fe700 1 -- 172.21.15.84:0/2305878727 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764012430 con 0x7f676c0cfc60 2023-07-06T21:43:05.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.434+0000 7f67727ff700 1 -- 172.21.15.84:0/2305878727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0cfc60 msgr2=0x7f676c167010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.443 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.434+0000 7f67727ff700 1 --2- 172.21.15.84:0/2305878727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0cfc60 0x7f676c167010 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6764009dc0 tx=0x7f676400b330 comp rx=0 tx=0).stop 2023-07-06T21:43:05.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.435+0000 7f67727ff700 1 -- 172.21.15.84:0/2305878727 shutdown_connections 2023-07-06T21:43:05.444 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.435+0000 7f67727ff700 1 --2- 172.21.15.84:0/2305878727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0cfc60 0x7f676c167010 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.435+0000 7f67727ff700 1 -- 172.21.15.84:0/2305878727 >> 172.21.15.84:0/2305878727 conn(0x7f676c06b350 msgr2=0x7f676c06b750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:05.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.435+0000 7f67727ff700 1 -- 172.21.15.84:0/2305878727 shutdown_connections 2023-07-06T21:43:05.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.435+0000 7f67727ff700 1 -- 172.21.15.84:0/2305878727 wait complete. 2023-07-06T21:43:05.445 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.436+0000 7f67727ff700 1 Processor -- start 2023-07-06T21:43:05.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 -- start start 2023-07-06T21:43:05.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 0x7f676c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.446 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c0cfc60 0x7f676c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0d3710 0x7f676c1f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c15cf60 con 0x7f676c0b83c0 2023-07-06T21:43:05.447 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c14a490 con 0x7f676c0cfc60 2023-07-06T21:43:05.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.437+0000 7f67727ff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f676c15d930 con 0x7f676c0d3710 2023-07-06T21:43:05.448 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 0x7f676c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 0x7f676c149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38838/0 (socket says 172.21.15.84:38838) 2023-07-06T21:43:05.449 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 -- 172.21.15.84:0/1984008888 learned_addr learned my addr 172.21.15.84:0/1984008888 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:05.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 -- 172.21.15.84:0/1984008888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0d3710 msgr2=0x7f676c1f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.450 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0d3710 0x7f676c1f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 -- 172.21.15.84:0/1984008888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c0cfc60 msgr2=0x7f676c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c0cfc60 0x7f676c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 -- 172.21.15.84:0/1984008888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6764009a70 con 0x7f676c0b83c0 2023-07-06T21:43:05.451 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f676bfff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 0x7f676c149f50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c680 tx=0x7f675c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c005d70 con 0x7f676c0b83c0 2023-07-06T21:43:05.452 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675c007070 con 0x7f676c0b83c0 2023-07-06T21:43:05.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.438+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c007420 con 0x7f676c0b83c0 2023-07-06T21:43:05.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.439+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c15c930 con 0x7f676c0b83c0 2023-07-06T21:43:05.453 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.439+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c1f2d30 con 0x7f676c0b83c0 2023-07-06T21:43:05.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.440+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c15b470 con 0x7f676c0b83c0 2023-07-06T21:43:05.454 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.441+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f675c007580 con 0x7f676c0b83c0 2023-07-06T21:43:05.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.442+0000 7f67697fa700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674c06ace0 0x7f674c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.442+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f675c088660 con 0x7f676c0b83c0 2023-07-06T21:43:05.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.442+0000 7f6763fff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674c06ace0 0x7f674c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.443+0000 7f6763fff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674c06ace0 0x7f674c06d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6754007fd0 tx=0x7f67540079a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.455 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.446+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f675c053990 con 0x7f676c0b83c0 2023-07-06T21:43:05.645 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.643+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f676c0b5430 con 0x7f674c06ace0 2023-07-06T21:43:05.646 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.643+0000 7f67697fa700 1 -- 172.21.15.84:0/1984008888 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f67540050d0 con 0x7f674c06ace0 2023-07-06T21:43:05.648 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674c06ace0 msgr2=0x7f674c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674c06ace0 0x7f674c06d190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6754007fd0 tx=0x7f67540079a0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.649 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 msgr2=0x7f676c149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 0x7f676c149f50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f675c00c680 tx=0x7f675c00ca40 comp rx=0 tx=0).stop 2023-07-06T21:43:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 shutdown_connections 2023-07-06T21:43:05.650 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f674c06ace0 0x7f674c06d190 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f676c0d3710 0x7f676c1f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f676c0b83c0 0x7f676c149f50 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 --2- 172.21.15.84:0/1984008888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f676c0cfc60 0x7f676c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 >> 172.21.15.84:0/1984008888 conn(0x7f676c06b350 msgr2=0x7f676c0b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:05.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 shutdown_connections 2023-07-06T21:43:05.652 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.646+0000 7f67727ff700 1 -- 172.21.15.84:0/1984008888 wait complete. 2023-07-06T21:43:05.652 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 6 to deep-scrub 2023-07-06T21:43:05.666 INFO:tasks.ceph:Scrubbing osd.7 2023-07-06T21:43:05.666 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-07-06T21:43:05.862 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.860+0000 7fb48e19d700 1 Processor -- start 2023-07-06T21:43:05.863 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.860+0000 7fb48e19d700 1 -- start start 2023-07-06T21:43:05.864 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48e19d700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fb4880d0c90 0x7fb4880b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48e19d700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4880b3e50 con 0x7fb4880d24e0 2023-07-06T21:43:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48e19d700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4880cfd40 con 0x7fb4880b2e40 2023-07-06T21:43:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48e19d700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48815ca50 con 0x7fb4880d0c90 2023-07-06T21:43:05.865 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48c99a700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fb4880d24e0 0x7fb4880d0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34096/0 (socket says 172.21.15.84:34096) 2023-07-06T21:43:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48c99a700 1 -- 172.21.15.84:0/1011466518 learned_addr learned my addr 172.21.15.84:0/1011466518 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:05.866 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.862+0000 7fb48d99c700 1 --2- 172.21.15.84:0/1011466518 >> v2:172.21.15.84:3300/0 conn(0x7fb4880d0c90 0x7fb4880b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb47ffff700 1 -- 172.21.15.84:0/1011466518 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211861985 0 0) 0x7fb4880b3e50 con 0x7fb4880d24e0 2023-07-06T21:43:05.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb47ffff700 1 -- 172.21.15.84:0/1011466518 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4740035b0 con 0x7fb4880d24e0 2023-07-06T21:43:05.867 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb48d99c700 1 -- 172.21.15.84:0/1011466518 >> v1:172.21.15.107:6789/0 conn(0x7fb4880b2e40 legacy=0x7fb4880cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb48d99c700 1 -- 172.21.15.84:0/1011466518 >> v1:172.21.15.84:6789/0 conn(0x7fb4880d24e0 legacy=0x7fb4880d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb48d99c700 1 -- 172.21.15.84:0/1011466518 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4880b3b70 con 0x7fb4880d0c90 2023-07-06T21:43:05.868 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb48d99c700 1 --2- 172.21.15.84:0/1011466518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d0c90 0x7fb4880b5800 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb480009d90 tx=0x7fb48000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5ddea290129494 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb47ffff700 1 -- 172.21.15.84:0/1011466518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48000e070 con 0x7fb4880d0c90 2023-07-06T21:43:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb47ffff700 1 -- 172.21.15.84:0/1011466518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48000bb20 con 0x7fb4880d0c90 2023-07-06T21:43:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.863+0000 7fb47ffff700 1 -- 172.21.15.84:0/1011466518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480012430 con 0x7fb4880d0c90 2023-07-06T21:43:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 -- 172.21.15.84:0/1011466518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d0c90 msgr2=0x7fb4880b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.869 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 --2- 172.21.15.84:0/1011466518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d0c90 0x7fb4880b5800 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fb480009d90 tx=0x7fb48000b330 comp rx=0 tx=0).stop 2023-07-06T21:43:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 -- 172.21.15.84:0/1011466518 shutdown_connections 2023-07-06T21:43:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 --2- 172.21.15.84:0/1011466518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d0c90 0x7fb4880b5800 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 -- 172.21.15.84:0/1011466518 >> 172.21.15.84:0/1011466518 conn(0x7fb48809bcf0 msgr2=0x7fb4880d29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 -- 172.21.15.84:0/1011466518 shutdown_connections 2023-07-06T21:43:05.870 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.864+0000 7fb48e19d700 1 -- 172.21.15.84:0/1011466518 wait complete. 2023-07-06T21:43:05.872 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.866+0000 7fb48e19d700 1 Processor -- start 2023-07-06T21:43:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.866+0000 7fb48e19d700 1 -- start start 2023-07-06T21:43:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48e19d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4880b2e40 0x7fb4880b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48e19d700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4880d0c90 0x7fb4881f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.873 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48e19d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d24e0 0x7fb4881f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48e19d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4881e69d0 con 0x7fb4880d24e0 2023-07-06T21:43:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48e19d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4881588f0 con 0x7fb4880b2e40 2023-07-06T21:43:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48e19d700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb488158bd0 con 0x7fb4880d0c90 2023-07-06T21:43:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d24e0 0x7fb4881f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.874 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d24e0 0x7fb4881f5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38848/0 (socket says 172.21.15.84:38848) 2023-07-06T21:43:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 -- 172.21.15.84:0/3531330412 learned_addr learned my addr 172.21.15.84:0/3531330412 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d19b700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4880b2e40 0x7fb4880b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48c99a700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4880d0c90 0x7fb4881f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 -- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4880d0c90 msgr2=0x7fb4881f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.875 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4880d0c90 0x7fb4881f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 -- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4880b2e40 msgr2=0x7fb4880b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4880b2e40 0x7fb4880b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48d99c700 1 -- 172.21.15.84:0/3531330412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009a70 con 0x7fb4880d24e0 2023-07-06T21:43:05.876 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.867+0000 7fb48c99a700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4880d0c90 0x7fb4881f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-06T21:43:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.868+0000 7fb48d99c700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d24e0 0x7fb4881f5b70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb488066ad0 tx=0x7fb480004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.868+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480012430 con 0x7fb4880d24e0 2023-07-06T21:43:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.868+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480020070 con 0x7fb4880d24e0 2023-07-06T21:43:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.868+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480004280 con 0x7fb4880d24e0 2023-07-06T21:43:05.877 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.868+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb488158eb0 con 0x7fb4880d24e0 2023-07-06T21:43:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.868+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4880b2270 con 0x7fb4880d24e0 2023-07-06T21:43:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.869+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4880cee30 con 0x7fb4880d24e0 2023-07-06T21:43:05.878 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.870+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fb4800043e0 con 0x7fb4880d24e0 2023-07-06T21:43:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.871+0000 7fb47dffb700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb46c06ace0 0x7fb46c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.879 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.871+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb480022070 con 0x7fb4880d24e0 2023-07-06T21:43:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.871+0000 7fb48d19b700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb46c06ace0 0x7fb46c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.871+0000 7fb47dffb700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fb46c071150 0x7fb46c073560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.871+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 --> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb46c073c30 con 0x7fb46c071150 2023-07-06T21:43:05.880 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.871+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb48008b780 con 0x7fb4880d24e0 2023-07-06T21:43:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.872+0000 7fb48d19b700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb46c06ace0 0x7fb46c06d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb484000fb0 tx=0x7fb484009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.872+0000 7fb48c99a700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fb46c071150 0x7fb46c073560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.872+0000 7fb48c99a700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fb46c071150 0x7fb46c073560 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:05.881 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.873+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== osd.7 v2:172.21.15.107:6824/440998536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb46c073c30 con 0x7fb46c071150 2023-07-06T21:43:05.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.888+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 --> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb4880ceb50 con 0x7fb46c071150 2023-07-06T21:43:05.891 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.889+0000 7fb47dffb700 1 -- 172.21.15.84:0/3531330412 <== osd.7 v2:172.21.15.107:6824/440998536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb478001870 con 0x7fb46c071150 2023-07-06T21:43:05.892 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2023-07-06T21:43:05.892 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-07-06T21:43:05.892 INFO:teuthology.orchestra.run.smithi084.stdout:} 2023-07-06T21:43:05.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.889+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fb46c071150 msgr2=0x7fb46c073560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.893 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.889+0000 7fb48e19d700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.107:6824/440998536,v1:172.21.15.107:6825/440998536] conn(0x7fb46c071150 0x7fb46c073560 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb46c06ace0 msgr2=0x7fb46c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fb46c06ace0 0x7fb46c06d190 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb484000fb0 tx=0x7fb484009040 comp rx=0 tx=0).stop 2023-07-06T21:43:05.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d24e0 msgr2=0x7fb4881f5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:05.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 --2- 172.21.15.84:0/3531330412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb4880d24e0 0x7fb4881f5b70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fb488066ad0 tx=0x7fb480004ad0 comp rx=0 tx=0).stop 2023-07-06T21:43:05.894 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48d99c700 1 -- 172.21.15.84:0/3531330412 reap_dead start 2023-07-06T21:43:05.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 shutdown_connections 2023-07-06T21:43:05.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 >> 172.21.15.84:0/3531330412 conn(0x7fb48809bcf0 msgr2=0x7fb4880b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:05.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 shutdown_connections 2023-07-06T21:43:05.895 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:05.890+0000 7fb48e19d700 1 -- 172.21.15.84:0/3531330412 wait complete. 2023-07-06T21:43:05.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-07-06T21:43:06.103 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.101+0000 7f8148fff700 1 Processor -- start 2023-07-06T21:43:06.104 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.102+0000 7f8148fff700 1 -- start start 2023-07-06T21:43:06.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f8148fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 0x7f8144070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f8148fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814415d930 con 0x7f8144159130 2023-07-06T21:43:06.105 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f8148fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8144070bc0 con 0x7f81440c9c10 2023-07-06T21:43:06.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f8148fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814415cd20 con 0x7f8144070230 2023-07-06T21:43:06.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f814259c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f8144159130 0x7f81440c4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34110/0 (socket says 172.21.15.84:34110) 2023-07-06T21:43:06.106 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f814259c700 1 -- 172.21.15.84:0/4221289253 learned_addr learned my addr 172.21.15.84:0/4221289253 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:06.108 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f813bfff700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 0x7f8144070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.103+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760639492 0 0) 0x7f814415d930 con 0x7f8144159130 2023-07-06T21:43:06.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81300035b0 con 0x7f8144159130 2023-07-06T21:43:06.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282617501 0 0) 0x7f813c001490 con 0x7f81440c9c10 2023-07-06T21:43:06.109 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f814415d930 con 0x7f81440c9c10 2023-07-06T21:43:06.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 74548156 0 0) 0x7f81300035b0 con 0x7f8144159130 2023-07-06T21:43:06.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 msgr2=0x7f8144070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:06.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 0x7f8144070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 >> v1:172.21.15.107:6789/0 conn(0x7f81440c9c10 legacy=0x7f81440ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.110 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 --> v1:172.21.15.84:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814415d000 con 0x7f8144159130 2023-07-06T21:43:06.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.0 v1:172.21.15.84:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1590645089 0 0) 0x7f8134001670 con 0x7f8144159130 2023-07-06T21:43:06.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 >> v1:172.21.15.84:6789/0 conn(0x7f8144159130 legacy=0x7f81440c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f8130003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.111 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 0x7f8130004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130004d90 0x7f8130005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814415cd20 con 0x7f8130003bc0 2023-07-06T21:43:06.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813000dcd0 con 0x7f8130004d90 2023-07-06T21:43:06.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813000de40 con 0x7f8144070230 2023-07-06T21:43:06.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f814259c700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f8130003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.112 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f813bfff700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 0x7f8130004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.104+0000 7f8142d9d700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130004d90 0x7f8130005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f814259c700 1 -- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 msgr2=0x7f8130004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:06.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f814259c700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144070230 0x7f8130004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f814259c700 1 -- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130004d90 msgr2=0x7f8130005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:06.113 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f814259c700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130004d90 0x7f8130005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f814259c700 1 -- 172.21.15.84:0/4221289253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300035b0 con 0x7f8130003bc0 2023-07-06T21:43:06.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f8142d9d700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130004d90 0x7f8130005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:06.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f814259c700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f8130003fb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f813400c210 tx=0x7f8130005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885235a07936f945 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:06.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400cd50 con 0x7f8130003bc0 2023-07-06T21:43:06.114 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134001af0 con 0x7f8130003bc0 2023-07-06T21:43:06.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f8141d9b700 1 -- 172.21.15.84:0/4221289253 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134005ba0 con 0x7f8130003bc0 2023-07-06T21:43:06.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f8148fff700 1 -- 172.21.15.84:0/4221289253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 msgr2=0x7f8130003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.105+0000 7f8148fff700 1 --2- 172.21.15.84:0/4221289253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f8130003fb0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f813400c210 tx=0x7f8130005880 comp rx=0 tx=0).stop 2023-07-06T21:43:06.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.106+0000 7f8142d9d700 1 -- 172.21.15.84:0/4221289253 reap_dead start 2023-07-06T21:43:06.115 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.106+0000 7f8148fff700 1 -- 172.21.15.84:0/4221289253 shutdown_connections 2023-07-06T21:43:06.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.106+0000 7f8148fff700 1 -- 172.21.15.84:0/4221289253 >> 172.21.15.84:0/4221289253 conn(0x7f81440cf3d0 msgr2=0x7f81440cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:06.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.106+0000 7f8148fff700 1 -- 172.21.15.84:0/4221289253 shutdown_connections 2023-07-06T21:43:06.116 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.106+0000 7f8148fff700 1 -- 172.21.15.84:0/4221289253 wait complete. 2023-07-06T21:43:06.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.107+0000 7f8148fff700 1 Processor -- start 2023-07-06T21:43:06.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 -- start start 2023-07-06T21:43:06.118 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f81441f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8144151b50 0x7f8144151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144159730 0x7f8144159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81441f29d0 con 0x7f8130003bc0 2023-07-06T21:43:06.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8144150ab0 con 0x7f8144151b50 2023-07-06T21:43:06.119 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.108+0000 7f8148fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8144150d90 con 0x7f8144159730 2023-07-06T21:43:06.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f81441f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f81441f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38870/0 (socket says 172.21.15.84:38870) 2023-07-06T21:43:06.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 -- 172.21.15.84:0/2199522048 learned_addr learned my addr 172.21.15.84:0/2199522048 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:06.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 -- 172.21.15.84:0/2199522048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144159730 msgr2=0x7f8144159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.120 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144159730 0x7f8144159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 -- 172.21.15.84:0/2199522048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8144151b50 msgr2=0x7f8144151fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8144151b50 0x7f8144151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 -- 172.21.15.84:0/2199522048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8144151070 con 0x7f8130003bc0 2023-07-06T21:43:06.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.109+0000 7f814259c700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f81441f2490 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f813400c800 tx=0x7f8134001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:06.121 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.110+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81340072d0 con 0x7f8130003bc0 2023-07-06T21:43:06.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.110+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134007430 con 0x7f8130003bc0 2023-07-06T21:43:06.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.110+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81340077c0 con 0x7f8130003bc0 2023-07-06T21:43:06.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.110+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8144151350 con 0x7f8130003bc0 2023-07-06T21:43:06.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.110+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81440cd610 con 0x7f8130003bc0 2023-07-06T21:43:06.122 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.111+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f814415c990 con 0x7f8130003bc0 2023-07-06T21:43:06.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.112+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f813401e430 con 0x7f8130003bc0 2023-07-06T21:43:06.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.113+0000 7f813affd700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f812406ace0 0x7f812406d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.123 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.113+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f813408a230 con 0x7f8130003bc0 2023-07-06T21:43:06.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.113+0000 7f813bfff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f812406ace0 0x7f812406d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.124 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.114+0000 7f813bfff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f812406ace0 0x7f812406d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f81440ac920 tx=0x7f812c009670 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:06.125 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.117+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f8134055710 con 0x7f8130003bc0 2023-07-06T21:43:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.315+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f81441f29d0 con 0x7f812406ace0 2023-07-06T21:43:06.317 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.315+0000 7f813affd700 1 -- 172.21.15.84:0/2199522048 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f81441f29d0 con 0x7f812406ace0 2023-07-06T21:43:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.317+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f812406ace0 msgr2=0x7f812406d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.320 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.317+0000 7f8148fff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f812406ace0 0x7f812406d190 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f81440ac920 tx=0x7f812c009670 comp rx=0 tx=0).stop 2023-07-06T21:43:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.317+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 msgr2=0x7f81441f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.317+0000 7f8148fff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f81441f2490 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f813400c800 tx=0x7f8134001980 comp rx=0 tx=0).stop 2023-07-06T21:43:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 shutdown_connections 2023-07-06T21:43:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f812406ace0 0x7f812406d190 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.321 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8144159730 0x7f8144159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8130003bc0 0x7f81441f2490 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 --2- 172.21.15.84:0/2199522048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8144151b50 0x7f8144151fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 >> 172.21.15.84:0/2199522048 conn(0x7f81440cf3d0 msgr2=0x7f81440b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:06.324 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 shutdown_connections 2023-07-06T21:43:06.325 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.318+0000 7f8148fff700 1 -- 172.21.15.84:0/2199522048 wait complete. 2023-07-06T21:43:06.325 INFO:teuthology.orchestra.run.smithi084.stderr:instructed osd(s) 7 to deep-scrub 2023-07-06T21:43:06.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:43:06.530 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.528+0000 7f7d7adff700 1 Processor -- start 2023-07-06T21:43:06.531 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.529+0000 7f7d7adff700 1 -- start start 2023-07-06T21:43:06.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7adff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d740b1260 0x7f7d740cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d740b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.532 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7adff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d74160960 con 0x7f7d7406ee90 2023-07-06T21:43:06.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7adff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7406a5b0 con 0x7f7d740b1260 2023-07-06T21:43:06.533 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d74160e90 con 0x7f7d7406ba90 2023-07-06T21:43:06.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d78b9b700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f7d740b1260 0x7f7d740cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.538 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7939c700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f7d7406ee90 0x7f7d7406f310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:34122/0 (socket says 172.21.15.84:34122) 2023-07-06T21:43:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d7939c700 1 -- 172.21.15.84:0/500838629 learned_addr learned my addr 172.21.15.84:0/500838629 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.530+0000 7f7d73fff700 1 --2- 172.21.15.84:0/500838629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d740b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d78b9b700 1 -- 172.21.15.84:0/500838629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 msgr2=0x7f7d740b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d78b9b700 1 --2- 172.21.15.84:0/500838629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d740b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.539 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d78b9b700 1 -- 172.21.15.84:0/500838629 >> v1:172.21.15.84:6789/0 conn(0x7f7d7406ee90 legacy=0x7f7d7406f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d78b9b700 1 -- 172.21.15.84:0/500838629 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7415c820 con 0x7f7d740b1260 2023-07-06T21:43:06.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d737fe700 1 -- 172.21.15.84:0/500838629 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872351968 0 0) 0x7f7d74160960 con 0x7f7d7406ee90 2023-07-06T21:43:06.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d78b9b700 1 --2- 172.21.15.84:0/500838629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d740cef60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009870 tx=0x7f7d68009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590bfaa9debcd514 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:06.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d737fe700 1 -- 172.21.15.84:0/500838629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68011830 con 0x7f7d740b1260 2023-07-06T21:43:06.540 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d737fe700 1 -- 172.21.15.84:0/500838629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d68011990 con 0x7f7d740b1260 2023-07-06T21:43:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.531+0000 7f7d737fe700 1 -- 172.21.15.84:0/500838629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68011d20 con 0x7f7d740b1260 2023-07-06T21:43:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 -- 172.21.15.84:0/500838629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 msgr2=0x7f7d740cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 --2- 172.21.15.84:0/500838629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d740cef60 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009870 tx=0x7f7d68009b80 comp rx=0 tx=0).stop 2023-07-06T21:43:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 -- 172.21.15.84:0/500838629 shutdown_connections 2023-07-06T21:43:06.541 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 --2- 172.21.15.84:0/500838629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d740cef60 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 --2- 172.21.15.84:0/500838629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d740b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 -- 172.21.15.84:0/500838629 >> 172.21.15.84:0/500838629 conn(0x7f7d740cc260 msgr2=0x7f7d740cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 -- 172.21.15.84:0/500838629 shutdown_connections 2023-07-06T21:43:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.532+0000 7f7d7adff700 1 -- 172.21.15.84:0/500838629 wait complete. 2023-07-06T21:43:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.534+0000 7f7d7adff700 1 Processor -- start 2023-07-06T21:43:06.542 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.534+0000 7f7d7adff700 1 -- start start 2023-07-06T21:43:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7adff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d741f14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7adff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d7406ee90 0x7f7d7415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7adff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d741e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7adff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d740c3700 con 0x7f7d740b1260 2023-07-06T21:43:06.543 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7adff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7415ee90 con 0x7f7d7406ee90 2023-07-06T21:43:06.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7adff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7415f170 con 0x7f7d7406ba90 2023-07-06T21:43:06.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d741e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d741e7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38876/0 (socket says 172.21.15.84:38876) 2023-07-06T21:43:06.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d78b9b700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d741f14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.544 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 -- 172.21.15.84:0/2361441266 learned_addr learned my addr 172.21.15.84:0/2361441266 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d73fff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d7406ee90 0x7f7d7415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 -- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 msgr2=0x7f7d741f14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d741f14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 -- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d7406ee90 msgr2=0x7f7d7415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.545 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d7406ee90 0x7f7d7415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.535+0000 7f7d7939c700 1 -- 172.21.15.84:0/2361441266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680094a0 con 0x7f7d740b1260 2023-07-06T21:43:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.536+0000 7f7d7939c700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d741e7bc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c002230 tx=0x7f7d6c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.536+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00aac0 con 0x7f7d740b1260 2023-07-06T21:43:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.536+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6c00ac40 con 0x7f7d740b1260 2023-07-06T21:43:06.546 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.536+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d740c3390 con 0x7f7d740b1260 2023-07-06T21:43:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.536+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0070d0 con 0x7f7d740b1260 2023-07-06T21:43:06.547 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.536+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7415ea30 con 0x7f7d740b1260 2023-07-06T21:43:06.548 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.537+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7415f3b0 con 0x7f7d740b1260 2023-07-06T21:43:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.538+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f7d6c007250 con 0x7f7d740b1260 2023-07-06T21:43:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.539+0000 7f7d717fa700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7d5806ace0 0x7f7d5806d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.539+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7d6c088cc0 con 0x7f7d740b1260 2023-07-06T21:43:06.549 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.539+0000 7f7d78b9b700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7d5806ace0 0x7f7d5806d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:06.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.540+0000 7f7d78b9b700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7d5806ace0 0x7f7d5806d190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7d680097c0 tx=0x7f7d68008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:06.550 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.543+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f7d7415f3b0 con 0x7f7d740b1260 2023-07-06T21:43:06.740 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.738+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7d740c3210 con 0x7f7d5806ace0 2023-07-06T21:43:06.743 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.741+0000 7f7d717fa700 1 -- 172.21.15.84:0/2361441266 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72561 (secure 0 0 0) 0x7f7d68011c00 con 0x7f7d5806ace0 2023-07-06T21:43:06.744 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:43:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.743+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7d5806ace0 msgr2=0x7f7d5806d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.743+0000 7f7d7adff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7d5806ace0 0x7f7d5806d190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7d680097c0 tx=0x7f7d68008030 comp rx=0 tx=0).stop 2023-07-06T21:43:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.743+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 msgr2=0x7f7d741e7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.743+0000 7f7d7adff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d741e7bc0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c002230 tx=0x7f7d6c00cf50 comp rx=0 tx=0).stop 2023-07-06T21:43:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 shutdown_connections 2023-07-06T21:43:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f7d5806ace0 0x7f7d5806d190 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7d740b1260 0x7f7d741e7bc0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d7406ba90 0x7f7d741f14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 --2- 172.21.15.84:0/2361441266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d7406ee90 0x7f7d7415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:06.747 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 >> 172.21.15.84:0/2361441266 conn(0x7f7d740cc260 msgr2=0x7f7d740b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:06.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 shutdown_connections 2023-07-06T21:43:06.748 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:06.744+0000 7f7d7adff700 1 -- 172.21.15.84:0/2361441266 wait complete. 2023-07-06T21:43:06.748 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:43:06.763 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2023-07-06T21:43:06.181015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222052,"kb_used_data":6436,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754752668,"statfs":{"total":773094113280,"available":772866732032,"internally_reserved":0,"allocated":6590464,"data_stored":3540189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.054319"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.578034+0000","last_change":"2023-07-06T21:39:20.918619+0000","last_active":"2023-07-06T21:42:56.578034+0000","last_peered":"2023-07-06T21:42:56.578034+0000","last_clean":"2023-07-06T21:42:56.578034+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:42:56.578034+0000","last_undegraded":"2023-07-06T21:42:56.578034+0000","last_fullsized":"2023-07-06T21:42:56.578034+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:02:13.288211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128555+0000","last_change":"2023-07-06T21:39:20.518391+0000","last_active":"2023-07-06T21:42:56.128555+0000","last_peered":"2023-07-06T21:42:56.128555+0000","last_clean":"2023-07-06T21:42:56.128555+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:42:56.128555+0000","last_undegraded":"2023-07-06T21:42:56.128555+0000","last_fullsized":"2023-07-06T21:42:56.128555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T04:26:57.209997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":78,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.837111+0000","last_change":"2023-07-06T21:39:20.813032+0000","last_active":"2023-07-06T21:42:56.837111+0000","last_peered":"2023-07-06T21:42:56.837111+0000","last_clean":"2023-07-06T21:42:56.837111+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:42:56.837111+0000","last_undegraded":"2023-07-06T21:42:56.837111+0000","last_fullsized":"2023-07-06T21:42:56.837111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:27:05.384819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.130367+0000","last_change":"2023-07-06T21:39:20.516198+0000","last_active":"2023-07-06T21:42:56.130367+0000","last_peered":"2023-07-06T21:42:56.130367+0000","last_clean":"2023-07-06T21:42:56.130367+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:42:56.130367+0000","last_undegraded":"2023-07-06T21:42:56.130367+0000","last_fullsized":"2023-07-06T21:42:56.130367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:23:36.326578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":230,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141735+0000","last_change":"2023-07-06T21:39:13.534959+0000","last_active":"2023-07-06T21:42:57.141735+0000","last_peered":"2023-07-06T21:42:57.141735+0000","last_clean":"2023-07-06T21:42:57.141735+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:42:57.141735+0000","last_undegraded":"2023-07-06T21:42:57.141735+0000","last_fullsized":"2023-07-06T21:42:57.141735+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:12.344816+0000","last_clean_scrub_stamp":"2023-07-06T21:39:12.344816+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:20:39.183176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.141546+0000","last_change":"2023-07-06T21:39:20.518885+0000","last_active":"2023-07-06T21:42:57.141546+0000","last_peered":"2023-07-06T21:42:57.141546+0000","last_clean":"2023-07-06T21:42:57.141546+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:42:57.141546+0000","last_undegraded":"2023-07-06T21:42:57.141546+0000","last_fullsized":"2023-07-06T21:42:57.141546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T22:01:48.302621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:57.140311+0000","last_change":"2023-07-06T21:39:20.918747+0000","last_active":"2023-07-06T21:42:57.140311+0000","last_peered":"2023-07-06T21:42:57.140311+0000","last_clean":"2023-07-06T21:42:57.140311+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:42:57.140311+0000","last_undegraded":"2023-07-06T21:42:57.140311+0000","last_fullsized":"2023-07-06T21:42:57.140311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T03:12:28.126913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.576736+0000","last_change":"2023-07-06T21:39:20.519932+0000","last_active":"2023-07-06T21:42:56.576736+0000","last_peered":"2023-07-06T21:42:56.576736+0000","last_clean":"2023-07-06T21:42:56.576736+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:42:56.576736+0000","last_undegraded":"2023-07-06T21:42:56.576736+0000","last_fullsized":"2023-07-06T21:42:56.576736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T09:02:07.005730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:42:56.128278+0000","last_change":"2023-07-06T21:39:20.522588+0000","last_active":"2023-07-06T21:42:56.128278+0000","last_peered":"2023-07-06T21:42:56.128278+0000","last_clean":"2023-07-06T21:42:56.128278+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:42:56.128278+0000","last_undegraded":"2023-07-06T21:42:56.128278+0000","last_fullsized":"2023-07-06T21:42:56.128278+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:39:19.497444+0000","last_clean_scrub_stamp":"2023-07-06T21:39:19.497444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T06:47:44.079675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1269760,"data_stored":885427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.53900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.101,"5min":1.28,"15min":1.28},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.995,"15min":0.995},"last":0.59799999999999998}]},{"osd":1,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.80600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89500000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.755}]},{"osd":3,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69699999999999995,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.872,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.86699999999999999}]},{"osd":4,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.972,"15min":1.972},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.379,"15min":1.379},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.88},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79500000000000004,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.80200000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85199999999999998,"5min":3.569,"15min":3.569},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.45300000000000001,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.76200000000000001,"15min":0.80300000000000005},"min":{"1min":0.48599999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83799999999999997,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.75800000000000001,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.99199999999999999,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.75700000000000001,"15min":0.80000000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82999999999999996,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.629}]},{"osd":3,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77900000000000003,"15min":0.81000000000000005},"min":{"1min":0.50800000000000001,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88500000000000001,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.80400000000000005,"15min":0.85299999999999998},"min":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.91800000000000004,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.64600000000000002,"15min":0.63400000000000001},"min":{"1min":0.501,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69099999999999995,"15min":0.69199999999999995},"min":{"1min":0.47799999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.86599999999999999,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.66800000000000004,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84199999999999997,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.72199999999999998,"15min":0.71399999999999997},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94599999999999995,"5min":1.081,"15min":1.081},"last":0.751}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90700000000000003,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72899999999999998,"15min":0.71999999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.90000000000000002,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.58299999999999996,"15min":0.63900000000000001},"min":{"1min":0.28000000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.58099999999999996,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.54900000000000004},"min":{"1min":0.34300000000000003,"5min":0.31,"15min":0.31},"max":{"1min":0.70599999999999996,"5min":2.246,"15min":2.246},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.52700000000000002,"15min":0.57299999999999995},"min":{"1min":0.313,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.48499999999999999,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.86499999999999999,"15min":0.94499999999999995},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.56599999999999995,"5min":11.913,"15min":11.913},"last":0.55100000000000005}]},{"osd":3,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":1.083,"15min":1.3},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.63900000000000001,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.80100000000000005,"15min":0.91900000000000004},"min":{"1min":0.36699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76500000000000001,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.443}]},{"osd":4,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.77600000000000002,"15min":0.81299999999999994},"min":{"1min":0.52100000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93899999999999995,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.84999999999999998,"15min":0.88400000000000001},"min":{"1min":0.55800000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98999999999999999,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.78100000000000003,"15min":0.80100000000000005},"min":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.92900000000000005,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.79800000000000004,"15min":0.83299999999999996},"min":{"1min":0.59199999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.028,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.875,"15min":0.89900000000000002},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":0.89600000000000002,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.81499999999999995,"15min":0.84299999999999997},"min":{"1min":0.55100000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0580000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:42:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.55700000000000005,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0089999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.749,"5min":0.83099999999999996,"15min":0.85999999999999999},"min":{"1min":0.58799999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.63800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":884408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48699999999999999,"15min":0.52900000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66600000000000004,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.498,"15min":0.53000000000000003},"min":{"1min":0.29899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61599999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.52700000000000002,"15min":0.56399999999999995},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.61299999999999999,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.53300000000000003,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63800000000000001,"5min":2.29,"15min":2.29},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.71399999999999997,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.439},{"interface":"front","average":{"1min":0.495,"5min":0.57999999999999996,"15min":0.62},"min":{"1min":0.39800000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69499999999999995,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.78500000000000003,"15min":0.82799999999999996},"min":{"1min":0.55300000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.95399999999999996,"5min":2.234,"15min":2.234},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.80100000000000005,"15min":0.84099999999999997},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.014,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.80400000000000005,"15min":0.84699999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0680000000000001,"5min":3.544,"15min":3.544},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.78900000000000003,"15min":0.81999999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":2.367,"15min":2.367},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.81499999999999995,"15min":0.85699999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.98899999999999999,"5min":2.395,"15min":2.395},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.80500000000000005,"15min":0.84599999999999997},"min":{"1min":0.54300000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96399999999999997,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jul 6 21:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.82099999999999995,"15min":0.85899999999999999},"min":{"1min":0.53200000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.998,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.85999999999999999,"15min":0.89600000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":3.669,"5min":3.669,"15min":3.669},"last":0.56599999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.40000000000000002,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.44500000000000001,"15min":0.434},"min":{"1min":0.36599999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.41099999999999998}]},{"osd":1,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":1.369,"15min":1.369},"last":0.44800000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.50600000000000001,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.63,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.38800000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.66100000000000003,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42899999999999999}]},{"osd":4,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.747,"15min":0.752},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.53400000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.857,"5min":1.857,"15min":1.857},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.56200000000000006,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.74399999999999999,"15min":0.76200000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1759999999999999,"5min":1.476,"15min":1.476},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0229999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.76300000000000001,"15min":0.76900000000000002},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.57499999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.82099999999999995,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1100000000000001,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.77800000000000002,"15min":0.79200000000000004},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.67300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64300000000000002,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.433,"5min":0.435,"15min":0.441},"min":{"1min":0.32800000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.53500000000000003,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.315}]},{"osd":1,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47699999999999998,"15min":0.47799999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51000000000000001,"15min":0.52800000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.67600000000000005,"5min":1.155,"15min":1.155},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.499,"15min":0.50600000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.65600000000000003,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52100000000000002,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.61299999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.41099999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.751,"15min":0.77500000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0449999999999999,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.292,"5min":1.304,"15min":1.304},"last":0.85799999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77300000000000002,"15min":0.78600000000000003},"min":{"1min":0.54300000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.252,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.755},{"interface":"front","average":{"1min":0.753,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.80200000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.2050000000000001,"5min":2.012,"15min":2.012},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76600000000000001,"15min":0.78000000000000003},"min":{"1min":0.46800000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.181,"5min":1.29,"15min":1.29},"last":0.82999999999999996}]},{"osd":7,"last update":"Thu Jul 6 21:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.52700000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1180000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.748,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.84699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.62},"min":{"1min":0.44700000000000001,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85699999999999998,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84399999999999997,"5min":1.871,"15min":1.871},"last":0.72099999999999997}]},{"osd":1,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.41499999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.80400000000000005,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.63200000000000001,"15min":0.622},"min":{"1min":0.46100000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83299999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.877,"5min":1.375,"15min":1.375},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67300000000000004,"15min":0.66500000000000004},"min":{"1min":0.53000000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79500000000000004,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.61699999999999999}]},{"osd":3,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.76600000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76600000000000001,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.90800000000000003,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.70699999999999996,"15min":0.72499999999999998},"min":{"1min":0.40600000000000003,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.875,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.51800000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90600000000000003,"5min":1.929,"15min":1.929},"last":0.57399999999999995}]},{"osd":6,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.72399999999999998,"15min":0.73599999999999999},"min":{"1min":0.48299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84699999999999998,"5min":2.859,"15min":2.859},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.46800000000000003,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96699999999999997,"5min":1.46,"15min":1.46},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jul 6 21:42:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.74099999999999999,"15min":0.751},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86299999999999999,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.626},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.77700000000000002},"min":{"1min":0.45400000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.98199999999999998,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.56000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574897,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66000000000000003,"15min":0.67800000000000005},"min":{"1min":0.498,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.85899999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.70699999999999996,"15min":0.69299999999999995},"min":{"1min":0.48399999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":3.4540000000000002,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.68200000000000005},"min":{"1min":0.52100000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.93500000000000005,"5min":1.038,"15min":1.038},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66700000000000004},"min":{"1min":0.47099999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.95299999999999996,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.66700000000000004,"15min":0.66500000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.91400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88800000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.77200000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.96299999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":3.5139999999999998,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.745}]},{"osd":4,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.46800000000000003,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89600000000000002,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70299999999999996,"15min":0.70499999999999996},"min":{"1min":0.53900000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.71299999999999997}]},{"osd":6,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.47599999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.88200000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.624},{"interface":"front","average":{"1min":0.75,"5min":0.749,"15min":0.75700000000000001},"min":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92200000000000004,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.73199999999999998}]},{"osd":7,"last update":"Thu Jul 6 21:42:12 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":0.99099999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.623,"5min":0.432,"15min":0.432},"max":{"1min":1.016,"5min":1.278,"15min":1.278},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:43:06.768 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.768 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.769 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.769 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.769 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-07-06T21:39:12.344816+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=12, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.769 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.770 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.770 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.770 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-07-06T21:39:19.497444+0000 time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=39, tm_sec=19, tm_wday=3, tm_yday=187, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=7, tm_mday=6, tm_hour=21, tm_min=42, tm_sec=59, tm_wday=3, tm_yday=187, tm_isdst=0) 2023-07-06T21:43:06.771 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-07-06T21:43:18.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:43:18.609 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:43:26.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-06T21:43:26.975 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.972+0000 7f958fbff700 1 Processor -- start 2023-07-06T21:43:26.976 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.972+0000 7f958fbff700 1 -- start start 2023-07-06T21:43:26.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.973+0000 7f958fbff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95880d3580 0x7f95880d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:26.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.973+0000 7f958fbff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880d0990 con 0x7f95880b8530 2023-07-06T21:43:26.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.973+0000 7f958fbff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880cd9d0 con 0x7f95880d13b0 2023-07-06T21:43:26.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.973+0000 7f958fbff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95880d3af0 con 0x7f95880d3580 2023-07-06T21:43:26.977 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d19a700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7f95880d3580 0x7f95880d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:26.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d99b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7f95880b8530 0x7f95880cee70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46284/0 (socket says 172.21.15.84:46284) 2023-07-06T21:43:26.978 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d99b700 1 -- 172.21.15.84:0/3495076927 learned_addr learned my addr 172.21.15.84:0/3495076927 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:26.979 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958c999700 1 -- 172.21.15.84:0/3495076927 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419415395 0 0) 0x7f95880d0990 con 0x7f95880b8530 2023-07-06T21:43:26.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d19a700 1 -- 172.21.15.84:0/3495076927 >> v1:172.21.15.107:6789/0 conn(0x7f95880d13b0 legacy=0x7f958815c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:26.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d19a700 1 -- 172.21.15.84:0/3495076927 >> v1:172.21.15.84:6789/0 conn(0x7f95880b8530 legacy=0x7f95880cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:26.980 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d19a700 1 -- 172.21.15.84:0/3495076927 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880cd410 con 0x7f95880d3580 2023-07-06T21:43:26.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958d19a700 1 --2- 172.21.15.84:0/3495076927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880d3580 0x7f95880d0e70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9578009840 tx=0x7f9578009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b4c8b7a6fb9bf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:26.981 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.974+0000 7f958c999700 1 -- 172.21.15.84:0/3495076927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578004230 con 0x7f95880d3580 2023-07-06T21:43:26.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958c999700 1 -- 172.21.15.84:0/3495076927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9578004390 con 0x7f95880d3580 2023-07-06T21:43:26.982 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958c999700 1 -- 172.21.15.84:0/3495076927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780036a0 con 0x7f95880d3580 2023-07-06T21:43:26.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 -- 172.21.15.84:0/3495076927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880d3580 msgr2=0x7f95880d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:26.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 --2- 172.21.15.84:0/3495076927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880d3580 0x7f95880d0e70 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9578009840 tx=0x7f9578009b50 comp rx=0 tx=0).stop 2023-07-06T21:43:26.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 -- 172.21.15.84:0/3495076927 shutdown_connections 2023-07-06T21:43:26.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 --2- 172.21.15.84:0/3495076927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880d3580 0x7f95880d0e70 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:26.983 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 -- 172.21.15.84:0/3495076927 >> 172.21.15.84:0/3495076927 conn(0x7f95880715c0 msgr2=0x7f9588071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:26.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 -- 172.21.15.84:0/3495076927 shutdown_connections 2023-07-06T21:43:26.984 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.975+0000 7f958fbff700 1 -- 172.21.15.84:0/3495076927 wait complete. 2023-07-06T21:43:26.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.977+0000 7f958fbff700 1 Processor -- start 2023-07-06T21:43:26.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.978+0000 7f958fbff700 1 -- start start 2023-07-06T21:43:26.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958fbff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 0x7f95881f1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:26.987 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958fbff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95880d13b0 0x7f95880c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:26.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958fbff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95880d3580 0x7f958815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:26.988 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958fbff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95881f1b80 con 0x7f95880b8530 2023-07-06T21:43:26.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958fbff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95880c7890 con 0x7f95880d3580 2023-07-06T21:43:26.989 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958fbff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880c7b70 con 0x7f95880d13b0 2023-07-06T21:43:26.990 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958d99b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 0x7f95881f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:26.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958d99b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 0x7f95881f1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54760/0 (socket says 172.21.15.84:54760) 2023-07-06T21:43:26.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958d99b700 1 -- 172.21.15.84:0/1073398962 learned_addr learned my addr 172.21.15.84:0/1073398962 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:26.991 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958e19c700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95880d3580 0x7f958815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:26.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.979+0000 7f958d19a700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95880d13b0 0x7f95880c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:26.992 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958d99b700 1 -- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95880d13b0 msgr2=0x7f95880c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:26.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958d99b700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95880d13b0 0x7f95880c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:26.993 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958d99b700 1 -- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95880d3580 msgr2=0x7f958815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:26.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958d99b700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95880d3580 0x7f958815bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:26.994 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958d99b700 1 -- 172.21.15.84:0/1073398962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780094a0 con 0x7f95880b8530 2023-07-06T21:43:26.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958d99b700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 0x7f95881f1640 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f95840010e0 tx=0x7f958400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:26.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840074e0 con 0x7f95880b8530 2023-07-06T21:43:26.995 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9584007660 con 0x7f95880b8530 2023-07-06T21:43:26.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95880c9e90 con 0x7f95880b8530 2023-07-06T21:43:26.996 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584005a70 con 0x7f95880b8530 2023-07-06T21:43:26.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.980+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95880ca450 con 0x7f95880b8530 2023-07-06T21:43:26.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.981+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95880c9cd0 con 0x7f95880b8530 2023-07-06T21:43:26.997 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.982+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7f9584005bd0 con 0x7f95880b8530 2023-07-06T21:43:26.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.983+0000 7f957e7fc700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f956c06ace0 0x7f956c06d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:26.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.983+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9584088670 con 0x7f95880b8530 2023-07-06T21:43:26.998 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.983+0000 7f958d19a700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f956c06ace0 0x7f956c06d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:26.999 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.984+0000 7f958d19a700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f956c06ace0 0x7f956c06d190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9578000c00 tx=0x7f957801c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.000 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:26.987+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7f9584053a30 con 0x7f95880b8530 2023-07-06T21:43:27.184 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.181+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 --> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95880d17f0 con 0x7f956c06ace0 2023-07-06T21:43:27.187 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.184+0000 7f957e7fc700 1 -- 172.21.15.84:0/1073398962 <== mgr.4105 v2:172.21.15.84:6832/4263456437 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72645 (secure 0 0 0) 0x7f95780041b0 con 0x7f956c06ace0 2023-07-06T21:43:27.188 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:43:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f956c06ace0 msgr2=0x7f956c06d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f956c06ace0 0x7f956c06d190 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9578000c00 tx=0x7f957801c040 comp rx=0 tx=0).stop 2023-07-06T21:43:27.190 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 msgr2=0x7f95881f1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 0x7f95881f1640 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f95840010e0 tx=0x7f958400c860 comp rx=0 tx=0).stop 2023-07-06T21:43:27.191 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 shutdown_connections 2023-07-06T21:43:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7f956c06ace0 0x7f956c06d190 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95880d3580 0x7f958815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95880b8530 0x7f95881f1640 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 --2- 172.21.15.84:0/1073398962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95880d13b0 0x7f95880c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.192 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 >> 172.21.15.84:0/1073398962 conn(0x7f95880715c0 msgr2=0x7f958815db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.186+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 shutdown_connections 2023-07-06T21:43:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.187+0000 7f958fbff700 1 -- 172.21.15.84:0/1073398962 wait complete. 2023-07-06T21:43:27.193 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2023-07-06T21:43:27.208 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2023-07-06T21:43:26.184897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222052,"kb_used_data":6436,"kb_used_omap":0,"kb_used_meta":215552,"kb_avail":754752668,"statfs":{"total":773094113280,"available":772866732032,"internally_reserved":0,"allocated":6590464,"data_stored":3540189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":220725248},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002404"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:02.503489+0000","last_change":"2023-07-06T21:43:02.503549+0000","last_active":"2023-07-06T21:43:02.503489+0000","last_peered":"2023-07-06T21:43:02.503489+0000","last_clean":"2023-07-06T21:43:02.503489+0000","last_became_active":"2023-07-06T21:39:20.918215+0000","last_became_peered":"2023-07-06T21:39:20.918215+0000","last_unstale":"2023-07-06T21:43:02.503489+0000","last_undegraded":"2023-07-06T21:43:02.503489+0000","last_fullsized":"2023-07-06T21:43:02.503489+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:02.503404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:02.503404+0000","last_clean_scrub_stamp":"2023-07-06T21:43:02.503404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:29:56.094248+0000","scrub_duration":0.001473373,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:05.559363+0000","last_change":"2023-07-06T21:43:05.559421+0000","last_active":"2023-07-06T21:43:05.559363+0000","last_peered":"2023-07-06T21:43:05.559363+0000","last_clean":"2023-07-06T21:43:05.559363+0000","last_became_active":"2023-07-06T21:39:20.518070+0000","last_became_peered":"2023-07-06T21:39:20.518070+0000","last_unstale":"2023-07-06T21:43:05.559363+0000","last_undegraded":"2023-07-06T21:43:05.559363+0000","last_fullsized":"2023-07-06T21:43:05.559363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:05.559224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:05.559224+0000","last_clean_scrub_stamp":"2023-07-06T21:43:05.559224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T02:33:09.060429+0000","scrub_duration":0.001771033,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:08.771127+0000","last_change":"2023-07-06T21:43:08.771213+0000","last_active":"2023-07-06T21:43:08.771127+0000","last_peered":"2023-07-06T21:43:08.771127+0000","last_clean":"2023-07-06T21:43:08.771127+0000","last_became_active":"2023-07-06T21:39:20.812706+0000","last_became_peered":"2023-07-06T21:39:20.812706+0000","last_unstale":"2023-07-06T21:43:08.771127+0000","last_undegraded":"2023-07-06T21:43:08.771127+0000","last_fullsized":"2023-07-06T21:43:08.771127+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:08.771000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:08.771000+0000","last_clean_scrub_stamp":"2023-07-06T21:43:08.771000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:50:15.614508+0000","scrub_duration":0.0018189879999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":93,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:06.589383+0000","last_change":"2023-07-06T21:43:06.589438+0000","last_active":"2023-07-06T21:43:06.589383+0000","last_peered":"2023-07-06T21:43:06.589383+0000","last_clean":"2023-07-06T21:43:06.589383+0000","last_became_active":"2023-07-06T21:39:20.515885+0000","last_became_peered":"2023-07-06T21:39:20.515885+0000","last_unstale":"2023-07-06T21:43:06.589383+0000","last_undegraded":"2023-07-06T21:43:06.589383+0000","last_fullsized":"2023-07-06T21:43:06.589383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:06.589292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:06.589292+0000","last_clean_scrub_stamp":"2023-07-06T21:43:06.589292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T01:00:08.233304+0000","scrub_duration":0.001693754,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"56'119","reported_seq":242,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:09.243701+0000","last_change":"2023-07-06T21:43:09.243701+0000","last_active":"2023-07-06T21:43:09.243701+0000","last_peered":"2023-07-06T21:43:09.243701+0000","last_clean":"2023-07-06T21:43:09.243701+0000","last_became_active":"2023-07-06T21:39:13.533778+0000","last_became_peered":"2023-07-06T21:39:13.533778+0000","last_unstale":"2023-07-06T21:43:09.243701+0000","last_undegraded":"2023-07-06T21:43:09.243701+0000","last_fullsized":"2023-07-06T21:43:09.243701+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"56'119","last_scrub_stamp":"2023-07-06T21:43:09.243530+0000","last_deep_scrub":"56'119","last_deep_scrub_stamp":"2023-07-06T21:43:09.243530+0000","last_clean_scrub_stamp":"2023-07-06T21:43:09.243530+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-08T00:36:20.560920+0000","scrub_duration":0.0085076140000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":84,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:10.211969+0000","last_change":"2023-07-06T21:43:10.211969+0000","last_active":"2023-07-06T21:43:10.211969+0000","last_peered":"2023-07-06T21:43:10.211969+0000","last_clean":"2023-07-06T21:43:10.211969+0000","last_became_active":"2023-07-06T21:39:20.518508+0000","last_became_peered":"2023-07-06T21:39:20.518508+0000","last_unstale":"2023-07-06T21:43:10.211969+0000","last_undegraded":"2023-07-06T21:43:10.211969+0000","last_fullsized":"2023-07-06T21:43:10.211969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:10.211824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:10.211824+0000","last_clean_scrub_stamp":"2023-07-06T21:43:10.211824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:24:18.746322+0000","scrub_duration":0.0027449330000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:12.228612+0000","last_change":"2023-07-06T21:43:12.228662+0000","last_active":"2023-07-06T21:43:12.228612+0000","last_peered":"2023-07-06T21:43:12.228612+0000","last_clean":"2023-07-06T21:43:12.228612+0000","last_became_active":"2023-07-06T21:39:20.918427+0000","last_became_peered":"2023-07-06T21:39:20.918427+0000","last_unstale":"2023-07-06T21:43:12.228612+0000","last_undegraded":"2023-07-06T21:43:12.228612+0000","last_fullsized":"2023-07-06T21:43:12.228612+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:12.228492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:12.228492+0000","last_clean_scrub_stamp":"2023-07-06T21:43:12.228492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:53:00.390698+0000","scrub_duration":0.002309546,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:04.499622+0000","last_change":"2023-07-06T21:43:04.499685+0000","last_active":"2023-07-06T21:43:04.499622+0000","last_peered":"2023-07-06T21:43:04.499622+0000","last_clean":"2023-07-06T21:43:04.499622+0000","last_became_active":"2023-07-06T21:39:20.519615+0000","last_became_peered":"2023-07-06T21:39:20.519615+0000","last_unstale":"2023-07-06T21:43:04.499622+0000","last_undegraded":"2023-07-06T21:43:04.499622+0000","last_fullsized":"2023-07-06T21:43:04.499622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:04.499528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:04.499528+0000","last_clean_scrub_stamp":"2023-07-06T21:43:04.499528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T21:52:51.587070+0000","scrub_duration":0.0020432860000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-07-06T21:43:07.840124+0000","last_change":"2023-07-06T21:43:07.840172+0000","last_active":"2023-07-06T21:43:07.840124+0000","last_peered":"2023-07-06T21:43:07.840124+0000","last_clean":"2023-07-06T21:43:07.840124+0000","last_became_active":"2023-07-06T21:39:20.522284+0000","last_became_peered":"2023-07-06T21:39:20.522284+0000","last_unstale":"2023-07-06T21:43:07.840124+0000","last_undegraded":"2023-07-06T21:43:07.840124+0000","last_fullsized":"2023-07-06T21:43:07.840124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-06T21:43:07.840005+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-06T21:43:07.840005+0000","last_clean_scrub_stamp":"2023-07-06T21:43:07.840005+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-07-07T23:18:09.407654+0000","scrub_duration":0.0016474199999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28192,"kb_used_data":1240,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343648,"statfs":{"total":96636764160,"available":96607895552,"internally_reserved":0,"allocated":1269760,"data_stored":885427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65000000000000002,"15min":0.65700000000000003},"min":{"1min":0.53900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":1.101,"5min":1.28,"15min":1.28},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79800000000000004,"5min":0.995,"15min":0.995},"last":0.73599999999999999}]},{"osd":1,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.47799999999999998,"5min":0.371,"15min":0.371},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.65400000000000003,"15min":0.66400000000000003},"min":{"1min":0.46100000000000002,"5min":0.441,"15min":0.441},"max":{"1min":0.80600000000000005,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63700000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79900000000000004,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70299999999999996,"15min":0.70599999999999996},"min":{"1min":0.51200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89500000000000002,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.75800000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69699999999999995,"15min":0.71199999999999997},"min":{"1min":0.50900000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.872,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72099999999999997,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95999999999999996,"5min":1.325,"15min":1.325},"last":0.80600000000000005}]},{"osd":4,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.65700000000000003,"15min":0.65900000000000003},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.92000000000000004,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.46300000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.94599999999999995,"5min":1.972,"15min":1.972},"last":0.82299999999999995}]},{"osd":5,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.75,"15min":0.76100000000000001},"min":{"1min":0.46800000000000003,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.1459999999999999,"5min":1.379,"15min":1.379},"last":0.62},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.74199999999999999,"15min":0.748},"min":{"1min":0.55900000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jul 6 21:42:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77600000000000002,"15min":0.78200000000000003},"min":{"1min":0.63700000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.89300000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79500000000000004,"15min":0.81899999999999995},"min":{"1min":0.56599999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.90700000000000003,"5min":1.9279999999999999,"15min":1.9279999999999999},"last":0.71599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71099999999999997,"15min":0.73399999999999999},"min":{"1min":0.48399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.872,"5min":3.569,"15min":3.569},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.45700000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.51000000000000001}]},{"osd":1,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64600000000000002,"15min":0.63700000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.73699999999999999,"15min":0.79500000000000004},"min":{"1min":0.49199999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98199999999999998,"5min":3.6779999999999999,"15min":3.6779999999999999},"last":0.71699999999999997}]},{"osd":2,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73199999999999998,"15min":0.79500000000000004},"min":{"1min":0.52200000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90900000000000003,"5min":3.7970000000000002,"15min":3.7970000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72799999999999998,"15min":0.79000000000000004},"min":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91200000000000003,"5min":3.6400000000000001,"15min":3.6400000000000001},"last":0.64600000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.75600000000000001,"15min":0.80200000000000005},"min":{"1min":0.54200000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94899999999999995,"5min":3.6200000000000001,"15min":3.6200000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.77600000000000002,"15min":0.84399999999999997},"min":{"1min":0.55100000000000005,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96799999999999997,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.60399999999999998}]},{"osd":4,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.65700000000000003,"15min":0.63700000000000001},"min":{"1min":0.503,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90100000000000002,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68600000000000005,"15min":0.68999999999999995},"min":{"1min":0.5,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.84899999999999998,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65800000000000003}]},{"osd":5,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67200000000000004,"15min":0.66000000000000003},"min":{"1min":0.48399999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89800000000000002,"5min":1.081,"15min":1.081},"last":0.63100000000000001}]},{"osd":7,"last update":"Thu Jul 6 21:43:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.73599999999999999,"15min":0.74399999999999999},"min":{"1min":0.51900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.98399999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73399999999999999,"15min":0.72199999999999998},"min":{"1min":0.54600000000000004,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.53600000000000003,"15min":0.623},"min":{"1min":0.308,"5min":0.251,"15min":0.251},"max":{"1min":0.621,"5min":5.5129999999999999,"15min":5.5129999999999999},"last":0.621},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.5,"15min":0.54000000000000004},"min":{"1min":0.33000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.60099999999999998,"5min":2.246,"15min":2.246},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.49399999999999999,"15min":0.56200000000000006},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66800000000000004,"5min":3.2709999999999999,"15min":3.2709999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.76600000000000001,"15min":0.91200000000000003},"min":{"1min":0.36199999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.69299999999999995,"5min":11.913,"15min":11.913},"last":0.64900000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.90100000000000002,"15min":1.2390000000000001},"min":{"1min":0.375,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.61199999999999999,"5min":22.053000000000001,"15min":22.053000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.70199999999999996,"15min":0.88600000000000001},"min":{"1min":0.37,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.63,"5min":9.1920000000000002,"15min":9.1920000000000002},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.747,"15min":0.80300000000000005},"min":{"1min":0.505,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.91100000000000003,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.80800000000000005,"15min":0.87},"min":{"1min":0.55600000000000005,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.878,"5min":5.4790000000000001,"15min":5.4790000000000001},"last":0.72499999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.76000000000000001,"15min":0.79400000000000004},"min":{"1min":0.53300000000000003,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.93500000000000005,"5min":2.9729999999999999,"15min":2.9729999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.76700000000000002,"15min":0.82299999999999995},"min":{"1min":0.53600000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.876,"5min":3.2280000000000002,"15min":3.2280000000000002},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.83799999999999997,"15min":0.88600000000000001},"min":{"1min":0.57499999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.86699999999999999,"5min":5.4349999999999996,"15min":5.4349999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.77900000000000003,"15min":0.83099999999999996},"min":{"1min":0.52900000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.84699999999999998,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.80000000000000004,"15min":0.81799999999999995},"min":{"1min":0.57899999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.90500000000000003,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.80200000000000005,"15min":0.84999999999999998},"min":{"1min":0.57699999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94299999999999995,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":0.753}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1236,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1265664,"data_stored":884408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46000000000000002,"15min":0.52000000000000002},"min":{"1min":0.318,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.57199999999999995,"5min":2.7869999999999999,"15min":2.7869999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48599999999999999,"15min":0.52600000000000002},"min":{"1min":0.33300000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.61199999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.504,"15min":0.55600000000000005},"min":{"1min":0.35099999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.53900000000000003,"5min":2.3319999999999999,"15min":2.3319999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.51200000000000001,"15min":0.56200000000000006},"min":{"1min":0.375,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63100000000000001,"5min":2.29,"15min":2.29},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47099999999999997,"15min":0.48499999999999999},"min":{"1min":0.35399999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.70499999999999996,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55800000000000005,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68000000000000005,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76200000000000001,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.94699999999999995,"5min":2.234,"15min":2.234},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77600000000000002,"15min":0.83199999999999996},"min":{"1min":0.57199999999999995,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.95799999999999996,"5min":3.5859999999999999,"15min":3.5859999999999999},"last":0.76500000000000001}]},{"osd":5,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78000000000000003,"15min":0.83899999999999997},"min":{"1min":0.55900000000000005,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0629999999999999,"5min":3.544,"15min":3.544},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.751,"5min":0.77200000000000002,"15min":0.81399999999999995},"min":{"1min":0.53900000000000003,"5min":0.503,"15min":0.503},"max":{"1min":0.91100000000000003,"5min":2.367,"15min":2.367},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.84999999999999998},"min":{"1min":0.55900000000000005,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.95099999999999996,"5min":2.395,"15min":2.395},"last":0.747},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.78400000000000003,"15min":0.83899999999999997},"min":{"1min":0.56000000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97099999999999997,"5min":2.2029999999999998,"15min":2.2029999999999998},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jul 6 21:43:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.80000000000000004,"15min":0.85199999999999998},"min":{"1min":0.59599999999999997,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.99199999999999999,"5min":2.2629999999999999,"15min":2.2629999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.82299999999999995,"15min":0.88400000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.024,"5min":3.669,"15min":3.669},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.40300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53800000000000003,"5min":0.746,"15min":0.746},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.436},"min":{"1min":0.32600000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.60799999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.308,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56699999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.49099999999999999,"15min":0.496},"min":{"1min":0.38800000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69799999999999995,"5min":1.369,"15min":1.369},"last":0.67300000000000004}]},{"osd":3,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.39900000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.65500000000000003,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.60799999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.63900000000000001}]},{"osd":4,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.57399999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.998,"5min":1.895,"15min":1.895},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.73699999999999999,"15min":0.74099999999999999},"min":{"1min":0.52500000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.256,"5min":1.857,"15min":1.857},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.089,"5min":1.913,"15min":1.913},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73799999999999999,"15min":0.76000000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.101,"5min":1.476,"15min":1.476},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.76100000000000001,"15min":0.78800000000000003},"min":{"1min":0.55100000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.292,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.51200000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.087,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.71499999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:43:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.81000000000000005,"15min":0.78800000000000003},"min":{"1min":0.54200000000000004,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.133,"5min":8.3190000000000008,"15min":8.3190000000000008},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77000000000000002,"15min":0.78900000000000003},"min":{"1min":0.57599999999999996,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.046,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.89300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56699999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.43099999999999999,"15min":0.44},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.57399999999999995,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.41599999999999998}]},{"osd":1,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.375,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.58299999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.443},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.498,"15min":0.52400000000000002},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.60199999999999998,"5min":1.155,"15min":1.155},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.5,"15min":0.50600000000000001},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.63700000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.53900000000000003},"min":{"1min":0.40100000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.56200000000000006,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.499}]},{"osd":4,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.56100000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94599999999999995,"5min":1.8640000000000001,"15min":1.8640000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.55200000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88,"5min":1.304,"15min":1.304},"last":0.72399999999999998}]},{"osd":5,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76500000000000001,"15min":0.78300000000000003},"min":{"1min":0.51800000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.93500000000000005,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.58899999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.94399999999999995,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.70399999999999996}]},{"osd":6,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.58699999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92100000000000004,"5min":2.012,"15min":2.012},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.59099999999999997,"5min":0.434,"15min":0.434},"max":{"1min":0.89900000000000002,"5min":1.29,"15min":1.29},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jul 6 21:43:20 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.74299999999999999,"15min":0.77900000000000003},"min":{"1min":0.54700000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.89000000000000001,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.754,"15min":0.77200000000000002},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.872,"5min":1.147,"15min":1.147},"last":0.63100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63,"15min":0.622},"min":{"1min":0.441,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82699999999999996,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.65300000000000002,"15min":0.63700000000000001},"min":{"1min":0.52900000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.84299999999999997,"5min":1.871,"15min":1.871},"last":0.72299999999999998}]},{"osd":1,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.52600000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.872,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.50700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.873,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.49099999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89100000000000001,"5min":1.375,"15min":1.375},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.68000000000000005,"15min":0.66700000000000004},"min":{"1min":0.47599999999999998,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88300000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.749}]},{"osd":3,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.66600000000000004,"15min":0.68000000000000005},"min":{"1min":0.46999999999999997,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83899999999999997,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.755,"15min":0.77700000000000002},"min":{"1min":0.63400000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.92200000000000004,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.40999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90600000000000003,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.73199999999999998,"15min":0.749},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91800000000000004,"5min":1.929,"15min":1.929},"last":0.628}]},{"osd":6,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.71299999999999997,"15min":0.73199999999999998},"min":{"1min":0.47099999999999997,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.874,"5min":2.859,"15min":2.859},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.498,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0429999999999999,"5min":1.46,"15min":1.46},"last":0.498}]},{"osd":7,"last update":"Thu Jul 6 21:43:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.72299999999999998,"15min":0.745},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.872,"5min":2.1339999999999999,"15min":2.1339999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.76700000000000002,"15min":0.77400000000000002},"min":{"1min":0.49099999999999999,"5min":0.443,"15min":0.443},"max":{"1min":0.94399999999999995,"5min":2.2410000000000001,"15min":2.2410000000000001},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27612,"kb_used_data":660,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344228,"statfs":{"total":96636764160,"available":96608489472,"internally_reserved":0,"allocated":675840,"data_stored":295059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64800000000000002,"15min":0.67400000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.75800000000000001,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.69599999999999995,"15min":0.68899999999999995},"min":{"1min":0.48199999999999998,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.85299999999999998,"5min":3.4540000000000002,"15min":3.4540000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67000000000000004,"15min":0.68000000000000005},"min":{"1min":0.45500000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.872,"5min":1.038,"15min":1.038},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.48999999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.77400000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76000000000000001}]},{"osd":2,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.65700000000000003,"15min":0.66200000000000003},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.625},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68799999999999994,"15min":0.70599999999999996},"min":{"1min":0.496,"5min":0.443,"15min":0.443},"max":{"1min":0.82099999999999995,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.57499999999999996,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.84999999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.74099999999999999},"min":{"1min":0.51700000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90400000000000003,"5min":3.5139999999999998,"15min":3.5139999999999998},"last":0.56399999999999995}]},{"osd":4,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66500000000000004,"15min":0.67500000000000004},"min":{"1min":0.45700000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.84699999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.70399999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92000000000000004,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74399999999999999}]},{"osd":6,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.52100000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.93600000000000005,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.53000000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.89800000000000002,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.78800000000000003}]},{"osd":7,"last update":"Thu Jul 6 21:43:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.034,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54200000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.97599999999999998,"5min":1.278,"15min":1.278},"last":0.77200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-06T21:43:27.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-07-06T21:43:27.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.394+0000 7fdc53fff700 1 Processor -- start 2023-07-06T21:43:27.398 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.395+0000 7fdc53fff700 1 -- start start 2023-07-06T21:43:27.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc53fff700 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdc4c0cfd10 0x7fdc4c0d2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.399 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc53fff700 1 --2- >> v2:172.21.15.84:3300/0 conn(0x7fdc4c0b1a80 0x7fdc4c0d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc53fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4c0d1830 con 0x7fdc4c0b81c0 2023-07-06T21:43:27.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc53fff700 1 -- --> v2:172.21.15.84:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4c0cf330 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc53fff700 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4c0d1270 con 0x7fdc4c0cfd10 2023-07-06T21:43:27.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc51d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7fdc4c0b81c0 0x7fdc4c0cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46292/0 (socket says 172.21.15.84:46292) 2023-07-06T21:43:27.400 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc51d9b700 1 -- 172.21.15.84:0/1891570107 learned_addr learned my addr 172.21.15.84:0/1891570107 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:27.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc5259c700 1 --2- 172.21.15.84:0/1891570107 >> v2:172.21.15.84:3300/0 conn(0x7fdc4c0b1a80 0x7fdc4c0d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.401 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc5159a700 1 --2- 172.21.15.84:0/1891570107 >> v2:172.21.15.107:3300/0 conn(0x7fdc4c0cfd10 0x7fdc4c0d2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.402 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.396+0000 7fdc50d99700 1 -- 172.21.15.84:0/1891570107 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758726417 0 0) 0x7fdc4c0d1830 con 0x7fdc4c0b81c0 2023-07-06T21:43:27.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc50d99700 1 -- 172.21.15.84:0/1891570107 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc380035b0 con 0x7fdc4c0b81c0 2023-07-06T21:43:27.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc5259c700 1 -- 172.21.15.84:0/1891570107 >> v2:172.21.15.107:3300/0 conn(0x7fdc4c0cfd10 msgr2=0x7fdc4c0d2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-06T21:43:27.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc5259c700 1 --2- 172.21.15.84:0/1891570107 >> v2:172.21.15.107:3300/0 conn(0x7fdc4c0cfd10 0x7fdc4c0d2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.403 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc5259c700 1 -- 172.21.15.84:0/1891570107 >> v1:172.21.15.84:6789/0 conn(0x7fdc4c0b81c0 legacy=0x7fdc4c0cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc5259c700 1 -- 172.21.15.84:0/1891570107 --> v2:172.21.15.84:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c0d1550 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc5259c700 1 --2- 172.21.15.84:0/1891570107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c0d28c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0d0770 tx=0x7fdc4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324d78962f1a5441 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc50d99700 1 -- 172.21.15.84:0/1891570107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4400e070 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc50d99700 1 -- 172.21.15.84:0/1891570107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4400bb20 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.404 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.397+0000 7fdc50d99700 1 -- 172.21.15.84:0/1891570107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44012430 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 -- 172.21.15.84:0/1891570107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 msgr2=0x7fdc4c0d28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 --2- 172.21.15.84:0/1891570107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c0d28c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0d0770 tx=0x7fdc4400b330 comp rx=0 tx=0).stop 2023-07-06T21:43:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 -- 172.21.15.84:0/1891570107 shutdown_connections 2023-07-06T21:43:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 --2- 172.21.15.84:0/1891570107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c0d28c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.405 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 --2- 172.21.15.84:0/1891570107 >> v2:172.21.15.107:3300/0 conn(0x7fdc4c0cfd10 0x7fdc4c0d2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 -- 172.21.15.84:0/1891570107 >> 172.21.15.84:0/1891570107 conn(0x7fdc4c06b2b0 msgr2=0x7fdc4c06b680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 -- 172.21.15.84:0/1891570107 shutdown_connections 2023-07-06T21:43:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.398+0000 7fdc53fff700 1 -- 172.21.15.84:0/1891570107 wait complete. 2023-07-06T21:43:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.399+0000 7fdc53fff700 1 Processor -- start 2023-07-06T21:43:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 -- start start 2023-07-06T21:43:27.408 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c14e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc4c0b81c0 0x7fdc4c0b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc4c0cfd10 0x7fdc4c0b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1e3030 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1e2420 con 0x7fdc4c0cfd10 2023-07-06T21:43:27.409 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.400+0000 7fdc53fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4c1e2700 con 0x7fdc4c0b81c0 2023-07-06T21:43:27.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c14e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c14e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54780/0 (socket says 172.21.15.84:54780) 2023-07-06T21:43:27.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 -- 172.21.15.84:0/3528353937 learned_addr learned my addr 172.21.15.84:0/3528353937 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:27.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc5259c700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc4c0cfd10 0x7fdc4c0b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.410 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc5159a700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc4c0b81c0 0x7fdc4c0b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 -- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc4c0b81c0 msgr2=0x7fdc4c0b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc4c0b81c0 0x7fdc4c0b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 -- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc4c0cfd10 msgr2=0x7fdc4c0b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc4c0cfd10 0x7fdc4c0b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.411 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 -- 172.21.15.84:0/3528353937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc44009a70 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.401+0000 7fdc51d9b700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c14e140 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c4d0 tx=0x7fdc4800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.402+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48005d70 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.402+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480070d0 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.402+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4c1e29e0 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.412 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.402+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48007440 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.402+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4c14e680 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.403+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4c04e1d0 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.404+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7fdc480075a0 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.404+0000 7fdc427fc700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdc3006ace0 0x7fdc3006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.405+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdc480886e0 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.405+0000 7fdc5159a700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdc3006ace0 0x7fdc3006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.405+0000 7fdc5159a700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdc3006ace0 0x7fdc3006d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c15bfd0 tx=0x7fdc3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.414 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.409+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7fdc48053a30 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.606 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.603+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdc4c0b4f30 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.610 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.607+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdc48053a30 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.611 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.607+0000 7fdc427fc700 1 -- 172.21.15.84:0/3528353937 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdc48017020 con 0x7fdc4c0b1a80 2023-07-06T21:43:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.610+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdc3006ace0 msgr2=0x7fdc3006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.610+0000 7fdc53fff700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdc3006ace0 0x7fdc3006d190 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c15bfd0 tx=0x7fdc3c008040 comp rx=0 tx=0).stop 2023-07-06T21:43:27.614 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.610+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 msgr2=0x7fdc4c14e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.610+0000 7fdc53fff700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c14e140 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c4d0 tx=0x7fdc4800c890 comp rx=0 tx=0).stop 2023-07-06T21:43:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 shutdown_connections 2023-07-06T21:43:27.615 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7fdc3006ace0 0x7fdc3006d190 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc4c0cfd10 0x7fdc4c0b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc4c0b1a80 0x7fdc4c14e140 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 --2- 172.21.15.84:0/3528353937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc4c0b81c0 0x7fdc4c0b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.616 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 >> 172.21.15.84:0/3528353937 conn(0x7fdc4c06b2b0 msgr2=0x7fdc4c0d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 shutdown_connections 2023-07-06T21:43:27.617 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.611+0000 7fdc53fff700 1 -- 172.21.15.84:0/3528353937 wait complete. 2023-07-06T21:43:27.631 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-07-06T21:43:27.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-06T21:43:27.824 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.821+0000 7ff7b2fff700 1 Processor -- start 2023-07-06T21:43:27.825 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.821+0000 7ff7b2fff700 1 -- start start 2023-07-06T21:43:27.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b2fff700 1 -- --> v1:172.21.15.84:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac06a820 con 0x7ff7ac158830 2023-07-06T21:43:27.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b2fff700 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7ac0c3f80 con 0x7ff7ac069e90 2023-07-06T21:43:27.826 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac0c4260 con 0x7ff7ac0c99d0 2023-07-06T21:43:27.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b0d9b700 1 --1- >> v1:172.21.15.84:6789/0 conn(0x7ff7ac158830 0x7ff7ac0c4d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.84:6789/0 says I am v1:172.21.15.84:46306/0 (socket says 172.21.15.84:46306) 2023-07-06T21:43:27.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b0d9b700 1 -- 172.21.15.84:0/3612242913 learned_addr learned my addr 172.21.15.84:0/3612242913 (peer_addr_for_me v1:172.21.15.84:0/0) 2023-07-06T21:43:27.827 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7b159c700 1 --2- 172.21.15.84:0/3612242913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 <== mon.0 v1:172.21.15.84:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228194888 0 0) 0x7ff7a0001490 con 0x7ff7ac158830 2023-07-06T21:43:27.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 --> v1:172.21.15.84:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff79c0046c0 con 0x7ff7ac158830 2023-07-06T21:43:27.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746271817 0 0) 0x7ff7ac0c3f80 con 0x7ff7ac069e90 2023-07-06T21:43:27.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.823+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a0001490 con 0x7ff7ac069e90 2023-07-06T21:43:27.829 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7b159c700 1 -- 172.21.15.84:0/3612242913 >> v1:172.21.15.107:6789/0 conn(0x7ff7ac069e90 legacy=0x7ff7ac0c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7b159c700 1 -- 172.21.15.84:0/3612242913 >> v1:172.21.15.84:6789/0 conn(0x7ff7ac158830 legacy=0x7ff7ac0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 <== mon.0 v1:172.21.15.84:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3060311963 0 0) 0x7ff79c0046c0 con 0x7ff7ac158830 2023-07-06T21:43:27.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7b159c700 1 -- 172.21.15.84:0/3612242913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac0c4540 con 0x7ff7ac0c99d0 2023-07-06T21:43:27.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7b159c700 1 --2- 172.21.15.84:0/3612242913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06a2e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009870 tx=0x7ff7a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3becfda19886f2c server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.830 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0001b80 con 0x7ff7ac0c99d0 2023-07-06T21:43:27.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff7a4004030 con 0x7ff7ac0c99d0 2023-07-06T21:43:27.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.824+0000 7ff7ab7fe700 1 -- 172.21.15.84:0/3612242913 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a40043a0 con 0x7ff7ac0c99d0 2023-07-06T21:43:27.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 -- 172.21.15.84:0/3612242913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 msgr2=0x7ff7ac06a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/3612242913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06a2e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4009870 tx=0x7ff7a4009b80 comp rx=0 tx=0).stop 2023-07-06T21:43:27.831 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 -- 172.21.15.84:0/3612242913 shutdown_connections 2023-07-06T21:43:27.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/3612242913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06a2e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 -- 172.21.15.84:0/3612242913 >> 172.21.15.84:0/3612242913 conn(0x7ff7ac0cf3d0 msgr2=0x7ff7ac0cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:27.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 -- 172.21.15.84:0/3612242913 shutdown_connections 2023-07-06T21:43:27.832 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.825+0000 7ff7b2fff700 1 -- 172.21.15.84:0/3612242913 wait complete. 2023-07-06T21:43:27.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.826+0000 7ff7b2fff700 1 Processor -- start 2023-07-06T21:43:27.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.827+0000 7ff7b2fff700 1 -- start start 2023-07-06T21:43:27.834 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.827+0000 7ff7b2fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 0x7ff7ac0c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.827+0000 7ff7b2fff700 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b2fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7ac158830 0x7ff7ac06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b2fff700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac152940 con 0x7ff7ac069e90 2023-07-06T21:43:27.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b2fff700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac152c20 con 0x7ff7ac158830 2023-07-06T21:43:27.835 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b2fff700 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac152f00 con 0x7ff7ac0c99d0 2023-07-06T21:43:27.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 0x7ff7ac0c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 0x7ff7ac0c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54784/0 (socket says 172.21.15.84:54784) 2023-07-06T21:43:27.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 -- 172.21.15.84:0/170183371 learned_addr learned my addr 172.21.15.84:0/170183371 (peer_addr_for_me v2:172.21.15.84:0/0) 2023-07-06T21:43:27.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7abfff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.836 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b159c700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7ac158830 0x7ff7ac06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 -- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 msgr2=0x7ff7ac06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 -- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7ac158830 msgr2=0x7ff7ac06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:27.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7ac158830 0x7ff7ac06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:27.837 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 -- 172.21.15.84:0/170183371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a40094a0 con 0x7ff7ac069e90 2023-07-06T21:43:27.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7abfff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:27.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b159c700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7ac158830 0x7ff7ac06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-06T21:43:27.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.828+0000 7ff7b0d9b700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 0x7ff7ac0c7230 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0001ba0 tx=0x7ff7a000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.829+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a000ae40 con 0x7ff7ac069e90 2023-07-06T21:43:27.838 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.829+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff7a00070d0 con 0x7ff7ac069e90 2023-07-06T21:43:27.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.829+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ac1531e0 con 0x7ff7ac069e90 2023-07-06T21:43:27.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.829+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a00074a0 con 0x7ff7ac069e90 2023-07-06T21:43:27.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.829+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7ac0c7770 con 0x7ff7ac069e90 2023-07-06T21:43:27.839 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.830+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7ac153700 con 0x7ff7ac069e90 2023-07-06T21:43:27.840 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.831+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) v1 ==== 89037+0+0 (secure 0 0 0) 0x7ff7a0007600 con 0x7ff7ac069e90 2023-07-06T21:43:27.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.832+0000 7ff7a97fa700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff79006ace0 0x7ff79006d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-06T21:43:27.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.832+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff7a0089360 con 0x7ff7ac069e90 2023-07-06T21:43:27.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.832+0000 7ff7abfff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff79006ace0 0x7ff79006d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-06T21:43:27.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.833+0000 7ff7abfff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff79006ace0 0x7ff79006d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0cb380 tx=0x7ff798009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-06T21:43:27.841 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:27.836+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176884 (secure 0 0 0) 0x7ff7ac153700 con 0x7ff7ac069e90 2023-07-06T21:43:28.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.073+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7ac06a820 con 0x7ff7ac069e90 2023-07-06T21:43:28.076 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.073+0000 7ff7a97fa700 1 -- 172.21.15.84:0/170183371 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3532 (secure 0 0 0) 0x7ff7ac06a820 con 0x7ff7ac069e90 2023-07-06T21:43:28.077 INFO:teuthology.orchestra.run.smithi084.stdout: 2023-07-06T21:43:28.077 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5324,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2976969551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6833","nonce":2976969551},{"type":"v1","addr":"172.21.15.107:6835","nonce":2976969551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6834/3248610324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3248610324},{"type":"v1","addr":"172.21.15.107:6834","nonce":3248610324}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5836,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6836/1384495799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1384495799},{"type":"v1","addr":"172.21.15.84:6836","nonce":1384495799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/726803293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6835","nonce":726803293},{"type":"v1","addr":"172.21.15.84:6837","nonce":726803293}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[]} 2023-07-06T21:43:28.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.075+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff79006ace0 msgr2=0x7ff79006d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:28.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.075+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff79006ace0 0x7ff79006d190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac0cb380 tx=0x7ff798009040 comp rx=0 tx=0).stop 2023-07-06T21:43:28.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.075+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 msgr2=0x7ff7ac0c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-06T21:43:28.079 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.075+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 0x7ff7ac0c7230 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0001ba0 tx=0x7ff7a000a340 comp rx=0 tx=0).stop 2023-07-06T21:43:28.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 shutdown_connections 2023-07-06T21:43:28.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:6832/4263456437,v1:172.21.15.84:6833/4263456437] conn(0x7ff79006ace0 0x7ff79006d190 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:28.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7ac158830 0x7ff7ac06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:28.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff7ac069e90 0x7ff7ac0c7230 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:28.080 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 --2- 172.21.15.84:0/170183371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7ac0c99d0 0x7ff7ac06dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-06T21:43:28.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 >> 172.21.15.84:0/170183371 conn(0x7ff7ac0cf3d0 msgr2=0x7ff7ac0ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-06T21:43:28.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 shutdown_connections 2023-07-06T21:43:28.081 INFO:teuthology.orchestra.run.smithi084.stderr:2023-07-06T21:43:28.076+0000 7ff7b2fff700 1 -- 172.21.15.84:0/170183371 wait complete. 2023-07-06T21:43:28.081 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 26 2023-07-06T21:43:28.097 DEBUG:tasks.cephfs.filesystem:already dead... 2023-07-06T21:43:28.097 INFO:teuthology.misc:Shutting down mds daemons... 2023-07-06T21:43:28.098 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-07-06T21:43:28.098 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.110 INFO:tasks.ceph.mds.a:Stopped 2023-07-06T21:43:28.111 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-07-06T21:43:28.111 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.165 INFO:tasks.ceph.mds.c:Stopped 2023-07-06T21:43:28.165 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-07-06T21:43:28.166 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.182 INFO:tasks.ceph.mds.b:Stopped 2023-07-06T21:43:28.183 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-07-06T21:43:28.183 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.240 INFO:tasks.ceph.mds.d:Stopped 2023-07-06T21:43:28.240 INFO:teuthology.misc:Shutting down osd daemons... 2023-07-06T21:43:28.241 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-07-06T21:43:28.241 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.282 INFO:tasks.ceph.osd.0:Stopped 2023-07-06T21:43:28.282 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-07-06T21:43:28.283 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.361 INFO:tasks.ceph.osd.1:Stopped 2023-07-06T21:43:28.362 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-07-06T21:43:28.362 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.438 INFO:tasks.ceph.osd.2:Stopped 2023-07-06T21:43:28.439 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-07-06T21:43:28.439 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.515 INFO:tasks.ceph.osd.3:Stopped 2023-07-06T21:43:28.516 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-07-06T21:43:28.517 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.565 INFO:tasks.ceph.osd.4:Stopped 2023-07-06T21:43:28.565 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-07-06T21:43:28.566 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.615 INFO:tasks.ceph.osd.5:Stopped 2023-07-06T21:43:28.616 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-07-06T21:43:28.616 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.667 INFO:tasks.ceph.osd.6:Stopped 2023-07-06T21:43:28.668 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-07-06T21:43:28.668 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.711 INFO:tasks.ceph.osd.7:Stopped 2023-07-06T21:43:28.712 INFO:teuthology.misc:Shutting down mgr daemons... 2023-07-06T21:43:28.713 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-07-06T21:43:28.713 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.775 INFO:tasks.ceph.mgr.y:Stopped 2023-07-06T21:43:28.776 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-07-06T21:43:28.777 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.835 INFO:tasks.ceph.mgr.x:Stopped 2023-07-06T21:43:28.836 INFO:teuthology.misc:Shutting down mon daemons... 2023-07-06T21:43:28.836 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-07-06T21:43:28.836 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.855 INFO:tasks.ceph.mon.a:Stopped 2023-07-06T21:43:28.856 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-07-06T21:43:28.856 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.874 INFO:tasks.ceph.mon.b:Stopped 2023-07-06T21:43:28.875 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-07-06T21:43:28.876 INFO:teuthology.orchestra.run:waiting for 300 2023-07-06T21:43:28.932 INFO:tasks.ceph.mon.c:Stopped 2023-07-06T21:43:28.933 INFO:tasks.ceph:Checking cluster log for badness... 2023-07-06T21:43:28.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-07-06T21:43:29.012 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:43:29.012 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-07-06T21:43:29.349 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:43:29.350 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-07-06T21:43:29.591 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:43:29.591 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-07-06T21:43:29.794 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:43:29.795 DEBUG:teuthology.orchestra.run.smithi084:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-07-06T21:43:30.041 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:43:30.042 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-07-06T21:43:30.555 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:43:30.556 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-07-06T21:43:30.751 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:43:30.752 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-07-06T21:43:30.953 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:43:30.954 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-07-06T21:43:31.138 INFO:tasks.ceph:Archiving mon data... 2023-07-06T21:43:31.139 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/data/mon.a.tgz 2023-07-06T21:43:31.140 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2023-07-06T21:43:31.187 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.O2aNcUFWMt 2023-07-06T21:43:31.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.O2aNcUFWMt 2023-07-06T21:43:31.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.O2aNcUFWMt 2023-07-06T21:43:31.458 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.O2aNcUFWMt is 363KB 2023-07-06T21:43:31.555 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.O2aNcUFWMt 2023-07-06T21:43:31.597 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/data/mon.b.tgz 2023-07-06T21:43:31.597 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2023-07-06T21:43:31.640 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.SWxuZx4TJh 2023-07-06T21:43:31.641 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.SWxuZx4TJh 2023-07-06T21:43:31.876 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.SWxuZx4TJh 2023-07-06T21:43:31.991 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.SWxuZx4TJh is 475KB 2023-07-06T21:43:32.086 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.SWxuZx4TJh 2023-07-06T21:43:32.130 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/data/mon.c.tgz 2023-07-06T21:43:32.130 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2023-07-06T21:43:32.211 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.5c2nJnvHOX 2023-07-06T21:43:32.212 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5c2nJnvHOX 2023-07-06T21:43:32.414 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.5c2nJnvHOX 2023-07-06T21:43:32.526 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.5c2nJnvHOX is 475KB 2023-07-06T21:43:32.623 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.5c2nJnvHOX 2023-07-06T21:43:32.664 INFO:tasks.ceph:Cleaning ceph cluster... 2023-07-06T21:43:32.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T21:43:32.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-07-06T21:43:32.770 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-07-06T21:43:32.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T21:43:32.775 DEBUG:teuthology.orchestra.run.smithi107:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-07-06T21:43:32.852 INFO:teuthology.orchestra.run.smithi107.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T21:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-07-06T21:43:32.871 INFO:tasks.ceph:Archiving crash dumps... 2023-07-06T21:43:32.873 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/remote/smithi084/crash 2023-07-06T21:43:32.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T21:43:32.942 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/remote/smithi107/crash 2023-07-06T21:43:32.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-07-06T21:43:33.009 INFO:tasks.ceph:Shutting down logrotate 2023-07-06T21:43:33.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:43:33.014 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-07-06T21:43:33.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T21:43:33.204 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-07-06T21:43:33.271 INFO:tasks.ceph:Compressing logs... 2023-07-06T21:43:33.271 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T21:43:33.276 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-06T21:43:33.337 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149526.log 2023-07-06T21:43:33.338 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143979.log 2023-07-06T21:43:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149526.log: gzip 88.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.149526.log.gz 2023-07-06T21:43:33.339 INFO:teuthology.orchestra.run.smithi084.stderr:--verbose -- /var/log/ceph/ceph-client.admin.153014.log 2023-07-06T21:43:33.340 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.149998.log 2023-07-06T21:43:33.341 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143979.log.gz 2023-07-06T21:43:33.341 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153014.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153014.log.gz 2023-07-06T21:43:33.341 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141930.log 2023-07-06T21:43:33.342 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-07-06T21:43:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130432.log 2023-07-06T21:43:33.343 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.149998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145401.log 2023-07-06T21:43:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T21:43:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141930.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141930.log.gz 2023-07-06T21:43:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155544.log 2023-07-06T21:43:33.345 INFO:teuthology.orchestra.run.smithi084.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.149998.log.gz 2023-07-06T21:43:33.345 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145309.log 2023-07-06T21:43:33.345 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145401.log.gz 2023-07-06T21:43:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155544.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155544.log.gz 2023-07-06T21:43:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140974.log 2023-07-06T21:43:33.347 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145309.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145309.log.gz 2023-07-06T21:43:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149093.log 2023-07-06T21:43:33.348 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130432.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.130432.log.gz 2023-07-06T21:43:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130151.log 2023-07-06T21:43:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-07-06T21:43:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130151.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.130151.log.gz 2023-07-06T21:43:33.349 INFO:teuthology.orchestra.run.smithi107.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-06T21:43:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130288.log 2023-07-06T21:43:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-07-06T21:43:33.350 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130288.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130288.log.gz 2023-07-06T21:43:33.351 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130104.log 2023-07-06T21:43:33.351 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130197.log 2023-07-06T21:43:33.352 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130104.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130104.log.gz 2023-07-06T21:43:33.353 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140974.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.140974.log.gz 2023-07-06T21:43:33.353 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153290.log 2023-07-06T21:43:33.353 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149093.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149093.log.gz 2023-07-06T21:43:33.353 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151411.log 2023-07-06T21:43:33.354 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138014.log 2023-07-06T21:43:33.354 INFO:teuthology.orchestra.run.smithi084.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.153290.log.gz 2023-07-06T21:43:33.354 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136753.log 2023-07-06T21:43:33.354 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151411.log.gz 2023-07-06T21:43:33.354 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138014.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138014.log.gz 2023-07-06T21:43:33.355 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142893.log 2023-07-06T21:43:33.355 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136753.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136753.log.gz 2023-07-06T21:43:33.355 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150353.log 2023-07-06T21:43:33.357 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T21:43:33.358 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130197.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130197.log.gz 2023-07-06T21:43:33.358 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130386.log 2023-07-06T21:43:33.358 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-07-06T21:43:33.358 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130386.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130386.log.gz 2023-07-06T21:43:33.358 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130243.log 2023-07-06T21:43:33.359 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.137460.log 2023-07-06T21:43:33.359 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T21:43:33.360 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152364.log 2023-07-06T21:43:33.360 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142893.log.gz 2023-07-06T21:43:33.360 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150353.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150353.log.gz 2023-07-06T21:43:33.361 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142411.log 2023-07-06T21:43:33.361 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152364.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152364.log.gz 2023-07-06T21:43:33.361 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147849.log 2023-07-06T21:43:33.361 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.142411.log.gz 2023-07-06T21:43:33.361 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155092.log 2023-07-06T21:43:33.362 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147849.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147849.log.gz 2023-07-06T21:43:33.362 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151641.log 2023-07-06T21:43:33.362 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155092.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.155092.log.gz 2023-07-06T21:43:33.362 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153198.log 2023-07-06T21:43:33.362 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151641.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151641.log.gz 2023-07-06T21:43:33.363 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137171.log 2023-07-06T21:43:33.364 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130243.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130243.log.gz 2023-07-06T21:43:33.365 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-07-06T21:43:33.365 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.3.137460.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-07-06T21:43:33.365 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.3.137460.log.gz 2023-07-06T21:43:33.365 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-07-06T21:43:33.365 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.136852.log 2023-07-06T21:43:33.367 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153198.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153198.log.gz 2023-07-06T21:43:33.367 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154008.log 2023-07-06T21:43:33.367 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137171.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.137171.log.gz 2023-07-06T21:43:33.367 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96583.log 2023-07-06T21:43:33.368 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154008.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154008.log.gz 2023-07-06T21:43:33.368 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130306.log 2023-07-06T21:43:33.368 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.tmp-client.admin.96583.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96583.log.gz 2023-07-06T21:43:33.368 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141424.log 2023-07-06T21:43:33.368 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130306.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.130306.log.gz 2023-07-06T21:43:33.369 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136437.log 2023-07-06T21:43:33.369 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141424.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141424.log.gz 2023-07-06T21:43:33.369 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157945.log 2023-07-06T21:43:33.369 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136437.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.136437.log.gz 2023-07-06T21:43:33.369 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158497.log 2023-07-06T21:43:33.370 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.1.136852.log: 91.1% -- replaced with /var/log/ceph/ceph-client.1.136852.log.gz 2023-07-06T21:43:33.371 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-07-06T21:43:33.371 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-07-06T21:43:33.371 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-07-06T21:43:33.372 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157945.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.157945.log.gz 2023-07-06T21:43:33.373 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144757.log 2023-07-06T21:43:33.373 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158497.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.158497.log.gz 2023-07-06T21:43:33.373 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-07-06T21:43:33.373 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144757.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144757.log.gz 2023-07-06T21:43:33.374 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145079.log 2023-07-06T21:43:33.374 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144665.log 2023-07-06T21:43:33.374 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145079.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145079.log.gz 2023-07-06T21:43:33.374 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136568.log 2023-07-06T21:43:33.375 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-mds.b.log: 91.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-07-06T21:43:33.375 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130333.log 2023-07-06T21:43:33.376 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144665.log: gzip -5 --verbose 87.7%-- /var/log/ceph/ceph-client.admin.156892.log -- replaced with /var/log/ceph/ceph-client.admin.144665.log.gz 2023-07-06T21:43:33.376 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-07-06T21:43:33.376 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136568.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136568.log.gz 2023-07-06T21:43:33.377 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144297.log 2023-07-06T21:43:33.377 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156892.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.156892.log.gz-5 --verbose 2023-07-06T21:43:33.377 INFO:teuthology.orchestra.run.smithi084.stderr:-- /var/log/ceph/ceph-client.admin.138965.log 2023-07-06T21:43:33.378 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.130333.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.130333.log.gz 2023-07-06T21:43:33.378 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141562.log 2023-07-06T21:43:33.379 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144297.log.gz 2023-07-06T21:43:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138965.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138965.log.gz 2023-07-06T21:43:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146578.log 2023-07-06T21:43:33.379 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141562.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141562.log.gz 2023-07-06T21:43:33.380 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157761.log 2023-07-06T21:43:33.380 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146578.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.146578.log.gz 2023-07-06T21:43:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141286.log 2023-07-06T21:43:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157761.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157761.log.gz 2023-07-06T21:43:33.381 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143370.log 2023-07-06T21:43:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141286.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141286.log.gz 2023-07-06T21:43:33.382 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157030.log 2023-07-06T21:43:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153060.log 2023-07-06T21:43:33.383 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143370.log.gz 2023-07-06T21:43:33.383 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157030.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157030.log.gz 2023-07-06T21:43:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142960.log 2023-07-06T21:43:33.384 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153060.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153060.log.gz 2023-07-06T21:43:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155203.log 2023-07-06T21:43:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142960.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142960.log.gz 2023-07-06T21:43:33.385 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152960.log 2023-07-06T21:43:33.386 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136987.log 2023-07-06T21:43:33.386 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155203.log.gz 2023-07-06T21:43:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152960.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152960.log.gz 2023-07-06T21:43:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143094.log 2023-07-06T21:43:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158755.log 2023-07-06T21:43:33.389 INFO:teuthology.orchestra.run.smithi084.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.136987.log.gz 2023-07-06T21:43:33.389 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142598.log 89.1% 2023-07-06T21:43:33.390 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.143094.log.gz 2023-07-06T21:43:33.391 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158755.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158755.log.gz 2023-07-06T21:43:33.391 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137643.log 2023-07-06T21:43:33.391 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142598.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142598.log.gz 2023-07-06T21:43:33.392 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146246.log 2023-07-06T21:43:33.392 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137643.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137643.log.gz 2023-07-06T21:43:33.393 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-07-06T21:43:33.393 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146246.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146246.log.gz 2023-07-06T21:43:33.394 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157168.log 2023-07-06T21:43:33.394 INFO:teuthology.orchestra.run.smithi107.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-07-06T21:43:33.395 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136113.log 2023-07-06T21:43:33.395 INFO:teuthology.orchestra.run.smithi084.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-07-06T21:43:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157168.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157168.log.gz 2023-07-06T21:43:33.396 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158405.log 2023-07-06T21:43:33.397 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136113.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136113.log.gz 2023-07-06T21:43:33.397 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138017.log 2023-07-06T21:43:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158405.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158405.log.gz 2023-07-06T21:43:33.398 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151705.log 2023-07-06T21:43:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138017.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138017.log.gz 2023-07-06T21:43:33.399 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142252.log 2023-07-06T21:43:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151705.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.151705.log.gz 2023-07-06T21:43:33.400 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147554.log 2023-07-06T21:43:33.401 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155249.log 2023-07-06T21:43:33.401 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142252.log.gz 2023-07-06T21:43:33.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147554.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147554.log.gz 2023-07-06T21:43:33.402 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149250.log 2023-07-06T21:43:33.403 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155249.log: gzip 88.4%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.155249.log.gz-- 2023-07-06T21:43:33.403 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148354.log 2023-07-06T21:43:33.404 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149250.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149250.log.gz 2023-07-06T21:43:33.404 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151365.log 2023-07-06T21:43:33.405 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148354.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148354.log.gz 2023-07-06T21:43:33.405 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144251.log 2023-07-06T21:43:33.406 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151365.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151365.log.gz 2023-07-06T21:43:33.406 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157807.log 2023-07-06T21:43:33.407 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144251.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144251.log.gz 2023-07-06T21:43:33.407 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.148692.log 2023-07-06T21:43:33.408 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157807.log.gz 2023-07-06T21:43:33.408 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158083.log 2023-07-06T21:43:33.409 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.148692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137645.log 2023-07-06T21:43:33.409 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158083.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.148692.log.gz 2023-07-06T21:43:33.410 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.158083.log.gz 2023-07-06T21:43:33.410 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155770.log 2023-07-06T21:43:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137645.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137645.log.gz 2023-07-06T21:43:33.411 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152028.log 2023-07-06T21:43:33.412 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155770.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.155770.log.gz 2023-07-06T21:43:33.412 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150602.log 2023-07-06T21:43:33.413 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152028.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152028.log.gz 2023-07-06T21:43:33.413 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158313.log 2023-07-06T21:43:33.414 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150602.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150602.log.gz 2023-07-06T21:43:33.414 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155590.log 2023-07-06T21:43:33.415 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158313.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158313.log.gz 2023-07-06T21:43:33.415 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149047.log 2023-07-06T21:43:33.416 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155590.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155590.log.gz 2023-07-06T21:43:33.416 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148006.log 2023-07-06T21:43:33.417 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149047.log.gz 2023-07-06T21:43:33.417 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144343.log 2023-07-06T21:43:33.418 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148006.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.148006.log.gz 2023-07-06T21:43:33.418 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156560.log 2023-07-06T21:43:33.419 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144343.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144343.log.gz 2023-07-06T21:43:33.419 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141332.log 2023-07-06T21:43:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156560.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156560.log.gz 2023-07-06T21:43:33.420 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152634.log 2023-07-06T21:43:33.421 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142068.log 2023-07-06T21:43:33.421 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141332.log.gz 2023-07-06T21:43:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152634.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.152634.log.gz 2023-07-06T21:43:33.422 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138374.log 2023-07-06T21:43:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142068.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142068.log.gz 2023-07-06T21:43:33.423 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156733.log 2023-07-06T21:43:33.424 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138374.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.138374.log.gz 2023-07-06T21:43:33.424 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142460.log 2023-07-06T21:43:33.425 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156733.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.156733.log.gz 2023-07-06T21:43:33.425 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153244.log 2023-07-06T21:43:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142460.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142460.log.gz 2023-07-06T21:43:33.426 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149660.log 2023-07-06T21:43:33.427 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153244.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153244.log.gz 2023-07-06T21:43:33.427 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139491.log 2023-07-06T21:43:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149660.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149660.log.gz 2023-07-06T21:43:33.428 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146012.log 2023-07-06T21:43:33.429 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139491.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.139491.log.gz 2023-07-06T21:43:33.429 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157122.log 2023-07-06T21:43:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146012.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146012.log.gz 2023-07-06T21:43:33.430 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-07-06T21:43:33.431 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157122.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157122.log.gz 2023-07-06T21:43:33.431 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151457.log 2023-07-06T21:43:33.432 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152166.log 2023-07-06T21:43:33.433 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151457.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.151457.log.gz 2023-07-06T21:43:33.433 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138919.log 2023-07-06T21:43:33.434 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152166.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.152166.log.gz 2023-07-06T21:43:33.435 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137447.log 2023-07-06T21:43:33.435 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138919.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138919.log.gz 2023-07-06T21:43:33.436 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136845.log 2023-07-06T21:43:33.437 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137447.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137447.log.gz 2023-07-06T21:43:33.437 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158451.log 2023-07-06T21:43:33.438 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136845.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136845.log.gz 2023-07-06T21:43:33.438 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151798.log 2023-07-06T21:43:33.439 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158451.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.158451.log.gz 2023-07-06T21:43:33.439 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149434.log 2023-07-06T21:43:33.440 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151798.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151798.log.gz 2023-07-06T21:43:33.440 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-07-06T21:43:33.441 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149434.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.149434.log.gz 2023-07-06T21:43:33.442 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157853.log 2023-07-06T21:43:33.442 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157260.log 2023-07-06T21:43:33.443 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157853.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157853.log.gz 2023-07-06T21:43:33.444 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138349.log 2023-07-06T21:43:33.445 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137648.log 2023-07-06T21:43:33.445 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.157260.log.gz 2023-07-06T21:43:33.446 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138349.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138349.log.gz 2023-07-06T21:43:33.447 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149480.log 2023-07-06T21:43:33.447 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137648.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137648.log.gz 2023-07-06T21:43:33.448 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158267.log 2023-07-06T21:43:33.449 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149480.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149480.log.gz 2023-07-06T21:43:33.450 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143232.log 2023-07-06T21:43:33.451 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158267.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158267.log.gz 2023-07-06T21:43:33.451 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151319.log 2023-07-06T21:43:33.452 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.143232.log.gz 2023-07-06T21:43:33.453 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143684.log 2023-07-06T21:43:33.454 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151319.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151319.log.gz 2023-07-06T21:43:33.454 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148909.log 2023-07-06T21:43:33.455 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143684.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.143684.log.gz 2023-07-06T21:43:33.456 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146624.log 2023-07-06T21:43:33.458 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.156108.log 2023-07-06T21:43:33.458 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148909.log.gz 2023-07-06T21:43:33.459 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146624.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.146624.log.gz 2023-07-06T21:43:33.459 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142298.log 2023-07-06T21:43:33.460 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.156108.log: 90.2% -- replaced with /var/log/ceph/ceph-client.0.156108.log.gz 2023-07-06T21:43:33.461 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158037.log 2023-07-06T21:43:33.462 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138022.log 2023-07-06T21:43:33.462 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142298.log.gz 2023-07-06T21:43:33.462 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158037.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158037.log.gz 2023-07-06T21:43:33.464 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136893.log 2023-07-06T21:43:33.465 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138022.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138022.log.gz 2023-07-06T21:43:33.465 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151982.log 2023-07-06T21:43:33.466 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136893.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136893.log.gz 2023-07-06T21:43:33.467 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-07-06T21:43:33.468 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151982.log: 90.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-07-06T21:43:33.469 INFO:teuthology.orchestra.run.smithi084.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.151982.log.gz 2023-07-06T21:43:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147803.log 2023-07-06T21:43:33.469 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147468.log 2023-07-06T21:43:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147803.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.147803.log.gz 2023-07-06T21:43:33.471 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141792.log 2023-07-06T21:43:33.472 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147468.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147468.log.gz 2023-07-06T21:43:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137217.log 2023-07-06T21:43:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141792.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.141792.log.gz 2023-07-06T21:43:33.473 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141240.log 2023-07-06T21:43:33.474 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137217.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137217.log.gz 2023-07-06T21:43:33.475 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147376.log 2023-07-06T21:43:33.476 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141240.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141240.log.gz 2023-07-06T21:43:33.476 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148190.log 2023-07-06T21:43:33.477 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147376.log.gz 2023-07-06T21:43:33.478 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146150.log 2023-07-06T21:43:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148190.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.148190.log.gz 2023-07-06T21:43:33.479 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144573.log 2023-07-06T21:43:33.480 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146150.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.146150.log.gz 2023-07-06T21:43:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129447.log 2023-07-06T21:43:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144573.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144573.log.gz 2023-07-06T21:43:33.482 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157490.log 2023-07-06T21:43:33.483 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.129447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138689.log 2023-07-06T21:43:33.484 INFO:teuthology.orchestra.run.smithi084.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129447.log.gz 2023-07-06T21:43:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157490.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157490.log.gz 2023-07-06T21:43:33.485 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158129.log 2023-07-06T21:43:33.486 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138689.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138689.log.gz 2023-07-06T21:43:33.487 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145874.log 2023-07-06T21:43:33.488 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158129.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158129.log.gz 2023-07-06T21:43:33.488 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144803.log 2023-07-06T21:43:33.489 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145874.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145874.log.gz 2023-07-06T21:43:33.489 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130439.log 2023-07-06T21:43:33.491 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144803.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144803.log.gz 2023-07-06T21:43:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148098.log 2023-07-06T21:43:33.492 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.130439.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130439.log.gz 2023-07-06T21:43:33.493 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150307.log 2023-07-06T21:43:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148098.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148098.log.gz 2023-07-06T21:43:33.494 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145669.log 2023-07-06T21:43:33.495 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150307.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.150307.log.gz 2023-07-06T21:43:33.496 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157352.log 2023-07-06T21:43:33.497 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145669.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145669.log.gz 2023-07-06T21:43:33.498 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150694.log 2023-07-06T21:43:33.499 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157352.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157352.log.gz 2023-07-06T21:43:33.500 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150491.log 2023-07-06T21:43:33.501 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157899.log 2023-07-06T21:43:33.501 INFO:teuthology.orchestra.run.smithi084.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150694.log.gz 2023-07-06T21:43:33.506 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150491.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150491.log.gz 2023-07-06T21:43:33.507 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148144.log 2023-07-06T21:43:33.507 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157398.log 2023-07-06T21:43:33.507 INFO:teuthology.orchestra.run.smithi084.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157899.log.gz 2023-07-06T21:43:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148144.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.148144.log.gz 2023-07-06T21:43:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152728.log 2023-07-06T21:43:33.508 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147422.log 2023-07-06T21:43:33.509 INFO:teuthology.orchestra.run.smithi084.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.157398.log.gz 2023-07-06T21:43:33.509 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152728.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152728.log.gz 2023-07-06T21:43:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144895.log 2023-07-06T21:43:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156779.log 2023-07-06T21:43:33.510 INFO:teuthology.orchestra.run.smithi084.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147422.log.gz 2023-07-06T21:43:33.510 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144895.log.gz 2023-07-06T21:43:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143186.log 2023-07-06T21:43:33.511 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138019.log 2023-07-06T21:43:33.512 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156779.log.gz 2023-07-06T21:43:33.512 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143186.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143186.log.gz 2023-07-06T21:43:33.513 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151273.log 2023-07-06T21:43:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138019.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.138019.log.gz 2023-07-06T21:43:33.514 INFO:teuthology.orchestra.run.smithi084.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.148237.log 2023-07-06T21:43:33.515 INFO:teuthology.orchestra.run.smithi107.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-07-06T21:43:33.515 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151273.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151273.log.gz 2023-07-06T21:43:33.516 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-07-06T21:43:33.517 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141746.log 2023-07-06T21:43:33.517 INFO:teuthology.orchestra.run.smithi084.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148237.log.gz 2023-07-06T21:43:33.518 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156938.log 2023-07-06T21:43:33.519 INFO:teuthology.orchestra.run.smithi084.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-07-06T21:43:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141746.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141746.log.gz 2023-07-06T21:43:33.520 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.154913.log 2023-07-06T21:43:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156938.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156938.log.gz 2023-07-06T21:43:33.521 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138827.log 2023-07-06T21:43:33.523 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.154913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137641.log 2023-07-06T21:43:33.524 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138827.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138827.log.gz 2023-07-06T21:43:33.524 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.154913.log.gz 2023-07-06T21:43:33.525 INFO:teuthology.orchestra.run.smithi107.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-07-06T21:43:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141516.log 2023-07-06T21:43:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137641.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137641.log.gz 2023-07-06T21:43:33.526 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141884.log 2023-07-06T21:43:33.527 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141194.log 2023-07-06T21:43:33.527 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141516.log.gz 2023-07-06T21:43:33.528 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141884.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141884.log.gz 2023-07-06T21:43:33.529 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152776.log 2023-07-06T21:43:33.530 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142114.log 2023-07-06T21:43:33.530 INFO:teuthology.orchestra.run.smithi084.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141194.log.gz 2023-07-06T21:43:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152776.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152776.log.gz 2023-07-06T21:43:33.531 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146532.log 2023-07-06T21:43:33.533 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142114.log: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-07-06T21:43:33.533 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151844.log 2023-07-06T21:43:33.533 INFO:teuthology.orchestra.run.smithi084.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142114.log.gz 2023-07-06T21:43:33.534 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146532.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146532.log.gz 2023-07-06T21:43:33.534 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142022.log 2023-07-06T21:43:33.535 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151844.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151844.log.gz 2023-07-06T21:43:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146762.log 2023-07-06T21:43:33.536 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142022.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.142022.log.gz 2023-07-06T21:43:33.537 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154054.log 2023-07-06T21:43:33.538 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152822.log 2023-07-06T21:43:33.539 INFO:teuthology.orchestra.run.smithi084.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.146762.log.gz 2023-07-06T21:43:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154054.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154054.log.gz 2023-07-06T21:43:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147895.log 2023-07-06T21:43:33.540 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152822.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152822.log.gz 2023-07-06T21:43:33.541 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147711.log 2023-07-06T21:43:33.542 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.147895.log.gz 2023-07-06T21:43:33.542 INFO:teuthology.orchestra.run.smithi107.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-07-06T21:43:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142709.log 2023-07-06T21:43:33.543 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147711.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147711.log.gz 2023-07-06T21:43:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157991.log 2023-07-06T21:43:33.544 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142709.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142709.log.gz 2023-07-06T21:43:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145920.log 2023-07-06T21:43:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157991.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.157991.log.gz 2023-07-06T21:43:33.546 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143048.log 2023-07-06T21:43:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145920.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.145920.log.gz 2023-07-06T21:43:33.547 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156606.log 2023-07-06T21:43:33.548 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143048.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143048.log.gz 2023-07-06T21:43:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143638.log 2023-07-06T21:43:33.549 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156606.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156606.log.gz 2023-07-06T21:43:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155295.log 2023-07-06T21:43:33.550 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143638.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143638.log.gz 2023-07-06T21:43:33.551 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155682.log 2023-07-06T21:43:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155295.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.155295.log.gz 2023-07-06T21:43:33.553 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-07-06T21:43:33.553 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144619.log 2023-07-06T21:43:33.553 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155682.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155682.log.gz 2023-07-06T21:43:33.554 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144163.log 2023-07-06T21:43:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144619.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144619.log.gz 2023-07-06T21:43:33.555 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138286.log 2023-07-06T21:43:33.556 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144163.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144163.log.gz 2023-07-06T21:43:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144117.log 2023-07-06T21:43:33.557 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138286.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138286.log.gz 2023-07-06T21:43:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152496.log 2023-07-06T21:43:33.558 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144117.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144117.log.gz 2023-07-06T21:43:33.559 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144849.log 2023-07-06T21:43:33.560 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152496.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152496.log.gz 2023-07-06T21:43:33.560 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151227.log 2023-07-06T21:43:33.561 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144849.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144849.log.gz 2023-07-06T21:43:33.562 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142160.log 2023-07-06T21:43:33.562 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151227.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.151227.log.gz 2023-07-06T21:43:33.563 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150941.log 2023-07-06T21:43:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142160.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142160.log.gz 2023-07-06T21:43:33.564 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-07-06T21:43:33.565 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157076.log 2023-07-06T21:43:33.565 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.150941.log.gz 2023-07-06T21:43:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.147170.log 2023-07-06T21:43:33.567 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157076.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.157076.log.gz 2023-07-06T21:43:33.567 INFO:teuthology.orchestra.run.smithi084.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-07-06T21:43:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151549.log 2023-07-06T21:43:33.568 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.147170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149614.log 2023-07-06T21:43:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151549.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151549.log.gz 2023-07-06T21:43:33.570 INFO:teuthology.orchestra.run.smithi084.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.147170.log.gz 2023-07-06T21:43:33.570 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155498.log 2023-07-06T21:43:33.571 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140927.log 2023-07-06T21:43:33.571 INFO:teuthology.orchestra.run.smithi084.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.149614.log.gz 2023-07-06T21:43:33.572 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155498.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.155498.log.gz 2023-07-06T21:43:33.573 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138004.log 2023-07-06T21:43:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140927.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140927.log.gz 2023-07-06T21:43:33.574 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152274.log 2023-07-06T21:43:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136661.log 2023-07-06T21:43:33.575 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138004.log.gz 2023-07-06T21:43:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152274.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152274.log.gz 2023-07-06T21:43:33.576 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150399.log 2023-07-06T21:43:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136661.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136661.log.gz 2023-07-06T21:43:33.577 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139241.log 2023-07-06T21:43:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150399.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150399.log.gz 2023-07-06T21:43:33.578 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156514.log 2023-07-06T21:43:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139241.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139241.log.gz 2023-07-06T21:43:33.579 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145447.log 2023-07-06T21:43:33.580 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-07-06T21:43:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156514.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.156514.log.gz 2023-07-06T21:43:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139011.log 2023-07-06T21:43:33.581 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145447.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145447.log.gz 2023-07-06T21:43:33.582 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151890.log 2023-07-06T21:43:33.583 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139011.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139011.log.gz 2023-07-06T21:43:33.583 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136707.log 2023-07-06T21:43:33.584 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151890.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.151890.log.gz 2023-07-06T21:43:33.584 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152074.log 2023-07-06T21:43:33.585 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148052.log 2023-07-06T21:43:33.585 INFO:teuthology.orchestra.run.smithi084.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136707.log.gz 2023-07-06T21:43:33.586 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152074.log.gz 2023-07-06T21:43:33.586 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149388.log 2023-07-06T21:43:33.587 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148052.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.148052.log.gz 2023-07-06T21:43:33.587 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138781.log 2023-07-06T21:43:33.588 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149388.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149388.log.gz 2023-07-06T21:43:33.588 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156422.log 2023-07-06T21:43:33.589 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138781.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138781.log.gz 2023-07-06T21:43:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149185.log 2023-07-06T21:43:33.590 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156422.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.156422.log.gz 2023-07-06T21:43:33.591 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143508.log 2023-07-06T21:43:33.591 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149185.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.149185.log.gz 2023-07-06T21:43:33.592 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158221.log 2023-07-06T21:43:33.593 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143508.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143508.log.gz 2023-07-06T21:43:33.593 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137263.log 2023-07-06T21:43:33.594 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158221.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158221.log.gz 2023-07-06T21:43:33.594 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138391.log 2023-07-06T21:43:33.595 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137263.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137263.log.gz 2023-07-06T21:43:33.595 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139103.log 2023-07-06T21:43:33.596 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138391.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138391.log.gz 2023-07-06T21:43:33.596 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139379.log 2023-07-06T21:43:33.597 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139103.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139103.log.gz 2023-07-06T21:43:33.598 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154192.log 2023-07-06T21:43:33.599 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140273.log 2023-07-06T21:43:33.599 INFO:teuthology.orchestra.run.smithi084.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.139379.log.gz 2023-07-06T21:43:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154192.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154192.log.gz 2023-07-06T21:43:33.600 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136238.log 2023-07-06T21:43:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.140273.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140273.log.gz 2023-07-06T21:43:33.601 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152120.log 2023-07-06T21:43:33.602 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136238.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.136238.log.gz 2023-07-06T21:43:33.603 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143933.log 2023-07-06T21:43:33.603 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152120.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152120.log.gz 2023-07-06T21:43:33.604 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155387.log 2023-07-06T21:43:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143933.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143933.log.gz 2023-07-06T21:43:33.605 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143868.log 2023-07-06T21:43:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155387.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155387.log.gz 2023-07-06T21:43:33.606 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143140.log 2023-07-06T21:43:33.607 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143868.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143868.log.gz 2023-07-06T21:43:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137079.log 2023-07-06T21:43:33.608 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143140.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143140.log.gz 2023-07-06T21:43:33.609 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151936.log 2023-07-06T21:43:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137079.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137079.log.gz 2023-07-06T21:43:33.610 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144941.log 2023-07-06T21:43:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151936.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151936.log.gz 2023-07-06T21:43:33.611 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152680.log 2023-07-06T21:43:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144941.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144941.log.gz 2023-07-06T21:43:33.612 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146348.log 2023-07-06T21:43:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152680.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152680.log.gz 2023-07-06T21:43:33.613 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137644.log 2023-07-06T21:43:33.614 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146348.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146348.log.gz 2023-07-06T21:43:33.615 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150445.log 2023-07-06T21:43:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137644.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137644.log.gz 2023-07-06T21:43:33.616 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145217.log 2023-07-06T21:43:33.617 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150445.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.150445.log.gz 2023-07-06T21:43:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154146.log 2023-07-06T21:43:33.618 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145217.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145217.log.gz 2023-07-06T21:43:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156468.log 2023-07-06T21:43:33.619 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154146.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.154146.log.gz 2023-07-06T21:43:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141700.log 2023-07-06T21:43:33.620 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156468.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156468.log.gz 2023-07-06T21:43:33.621 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150261.log 2023-07-06T21:43:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141700.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141700.log.gz 2023-07-06T21:43:33.622 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139057.log 2023-07-06T21:43:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150261.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.150261.log.gz 2023-07-06T21:43:33.623 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147665.log 2023-07-06T21:43:33.624 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139057.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.139057.log.gz 2023-07-06T21:43:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153106.log 2023-07-06T21:43:33.625 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147665.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.147665.log.gz 2023-07-06T21:43:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145966.log 2023-07-06T21:43:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153106.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153106.log.gz 2023-07-06T21:43:33.627 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157444.log 2023-07-06T21:43:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145966.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.145966.log.gz 2023-07-06T21:43:33.628 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145807.log 2023-07-06T21:43:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157444.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157444.log.gz 2023-07-06T21:43:33.629 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150556.log 2023-07-06T21:43:33.630 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145807.log.gz 2023-07-06T21:43:33.630 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-07-06T21:43:33.631 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-07-06T21:43:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150556.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150556.log.gz 2023-07-06T21:43:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141378.log 2023-07-06T21:43:33.632 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154441.log 2023-07-06T21:43:33.633 INFO:teuthology.orchestra.run.smithi107.stderr: 2023-07-06T21:43:33.634 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m0.318s 2023-07-06T21:43:33.634 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m1.169s 2023-07-06T21:43:33.634 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.196s 2023-07-06T21:43:33.635 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141378.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141378.log.gz 2023-07-06T21:43:33.635 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136941.log 2023-07-06T21:43:33.636 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154441.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154441.log.gz 2023-07-06T21:43:33.636 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139333.log 2023-07-06T21:43:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136941.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136941.log.gz 2023-07-06T21:43:33.637 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145033.log 2023-07-06T21:43:33.638 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139333.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.139333.log.gz 2023-07-06T21:43:33.639 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148955.log 2023-07-06T21:43:33.640 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154575.log 2023-07-06T21:43:33.640 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145033.log.gz 2023-07-06T21:43:33.641 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.148955.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.148955.log.gz 2023-07-06T21:43:33.641 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151181.log 2023-07-06T21:43:33.642 INFO:teuthology.orchestra.run.smithi084.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-07-06T21:43:33.643 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136301.log 2023-07-06T21:43:33.643 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.154575.log.gz 2023-07-06T21:43:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151181.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.151181.log.gz 2023-07-06T21:43:33.644 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142801.log 2023-07-06T21:43:33.645 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136301.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136301.log.gz 2023-07-06T21:43:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146198.log 2023-07-06T21:43:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142801.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142801.log.gz 2023-07-06T21:43:33.647 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149001.log 2023-07-06T21:43:33.648 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138873.log 2023-07-06T21:43:33.648 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146198.log.gz 2023-07-06T21:43:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149001.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149001.log.gz 2023-07-06T21:43:33.649 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150832.log 2023-07-06T21:43:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138873.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.138873.log.gz 2023-07-06T21:43:33.650 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143822.log 2023-07-06T21:43:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150832.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150832.log.gz 2023-07-06T21:43:33.651 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155341.log 2023-07-06T21:43:33.652 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143822.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143822.log.gz 2023-07-06T21:43:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149342.log 2023-07-06T21:43:33.653 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155341.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155341.log.gz 2023-07-06T21:43:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138003.log 2023-07-06T21:43:33.654 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149342.log: gzip -5 --verbose 89.0%-- /var/log/ceph/ceph-client.admin.152450.log -- replaced with /var/log/ceph/ceph-client.admin.149342.log.gz 2023-07-06T21:43:33.655 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-07-06T21:43:33.656 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138003.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138003.log.gz 2023-07-06T21:43:33.656 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145761.log 2023-07-06T21:43:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152450.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.152450.log.gz 2023-07-06T21:43:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139149.log 2023-07-06T21:43:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145761.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145761.log.gz 2023-07-06T21:43:33.658 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144071.log 2023-07-06T21:43:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139149.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139149.log.gz 2023-07-06T21:43:33.659 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153897.log 2023-07-06T21:43:33.660 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144071.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144071.log.gz 2023-07-06T21:43:33.661 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151089.log 2023-07-06T21:43:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153897.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.153897.log.gz 2023-07-06T21:43:33.662 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154487.log 2023-07-06T21:43:33.663 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151089.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.151089.log.gz 2023-07-06T21:43:33.663 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151595.log 2023-07-06T21:43:33.664 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154487.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154487.log.gz 2023-07-06T21:43:33.665 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144025.log 2023-07-06T21:43:33.665 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151595.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151595.log.gz 2023-07-06T21:43:33.666 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141470.log 2023-07-06T21:43:33.666 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144025.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144025.log.gz 2023-07-06T21:43:33.667 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137401.log 2023-07-06T21:43:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141470.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141470.log.gz 2023-07-06T21:43:33.668 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144481.log 2023-07-06T21:43:33.669 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137401.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.137401.log.gz 2023-07-06T21:43:33.669 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150215.log 2023-07-06T21:43:33.670 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144481.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144481.log.gz 2023-07-06T21:43:33.670 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138735.log 2023-07-06T21:43:33.671 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150215.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150215.log.gz 2023-07-06T21:43:33.672 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.140102.log 2023-07-06T21:43:33.672 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138735.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138735.log.gz 2023-07-06T21:43:33.673 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153360.log 2023-07-06T21:43:33.673 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.140102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157669.log 2023-07-06T21:43:33.674 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.140102.log.gz 2023-07-06T21:43:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153360.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153360.log.gz 2023-07-06T21:43:33.675 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142644.log 2023-07-06T21:43:33.676 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157669.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157669.log.gz 2023-07-06T21:43:33.676 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158175.log 2023-07-06T21:43:33.677 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142644.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142644.log.gz 2023-07-06T21:43:33.677 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136522.log 2023-07-06T21:43:33.678 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158175.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158175.log.gz 2023-07-06T21:43:33.678 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141148.log 2023-07-06T21:43:33.679 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136522.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136522.log.gz 2023-07-06T21:43:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149296.log 2023-07-06T21:43:33.680 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141148.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141148.log.gz 2023-07-06T21:43:33.681 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138387.log 2023-07-06T21:43:33.681 INFO:teuthology.orchestra.run.smithi084.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-07-06T21:43:33.682 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149296.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.149296.log.gz 2023-07-06T21:43:33.682 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156825.log 2023-07-06T21:43:33.683 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138387.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138387.log.gz 2023-07-06T21:43:33.684 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147960.log 2023-07-06T21:43:33.685 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142847.log 2023-07-06T21:43:33.685 INFO:teuthology.orchestra.run.smithi084.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156825.log.gz 2023-07-06T21:43:33.686 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147960.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.147960.log.gz-5 2023-07-06T21:43:33.686 INFO:teuthology.orchestra.run.smithi084.stderr:--verbose -- /var/log/ceph/ceph-client.admin.142755.log 2023-07-06T21:43:33.687 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142847.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142847.log.gz 2023-07-06T21:43:33.687 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155452.log 2023-07-06T21:43:33.688 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142755.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142755.log.gz 2023-07-06T21:43:33.688 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138021.log 2023-07-06T21:43:33.689 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155452.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155452.log.gz 2023-07-06T21:43:33.690 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145537.log 2023-07-06T21:43:33.690 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138021.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138021.log.gz 2023-07-06T21:43:33.691 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154100.log 2023-07-06T21:43:33.691 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145537.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145537.log.gz 2023-07-06T21:43:33.692 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151752.log 2023-07-06T21:43:33.692 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154100.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.154100.log.gz 2023-07-06T21:43:33.693 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146486.log 2023-07-06T21:43:33.694 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151752.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151752.log.gz 2023-07-06T21:43:33.694 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147619.log 2023-07-06T21:43:33.695 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146486.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146486.log.gz 2023-07-06T21:43:33.695 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138367.log 2023-07-06T21:43:33.696 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147619.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.147619.log.gz 2023-07-06T21:43:33.696 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141608.log 2023-07-06T21:43:33.697 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138367.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138367.log.gz 2023-07-06T21:43:33.698 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137033.log 2023-07-06T21:43:33.698 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141608.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.141608.log.gz 2023-07-06T21:43:33.699 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145355.log 2023-07-06T21:43:33.699 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137033.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.137033.log.gz 2023-07-06T21:43:33.700 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137654.log 2023-07-06T21:43:33.701 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145355.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.145355.log.gz 2023-07-06T21:43:33.701 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158709.log 2023-07-06T21:43:33.702 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137654.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137654.log.gz 2023-07-06T21:43:33.702 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158359.log 2023-07-06T21:43:33.703 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158709.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158709.log.gz 2023-07-06T21:43:33.703 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.153719.log 2023-07-06T21:43:33.704 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158359.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158359.log.gz 2023-07-06T21:43:33.704 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154303.log 2023-07-06T21:43:33.706 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.0.153719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145263.log 2023-07-06T21:43:33.706 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154303.log: 91.8% -- replaced with /var/log/ceph/ceph-client.0.153719.log.gz 2023-07-06T21:43:33.707 INFO:teuthology.orchestra.run.smithi084.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154303.log.gz 2023-07-06T21:43:33.707 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142552.log 2023-07-06T21:43:33.708 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145263.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145263.log.gz 2023-07-06T21:43:33.708 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137355.log 2023-07-06T21:43:33.709 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142552.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142552.log.gz 2023-07-06T21:43:33.709 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157306.log 2023-07-06T21:43:33.710 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137355.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137355.log.gz 2023-07-06T21:43:33.710 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150648.log 2023-07-06T21:43:33.711 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157306.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157306.log.gz 2023-07-06T21:43:33.712 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152542.log 2023-07-06T21:43:33.712 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150648.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150648.log.gz 2023-07-06T21:43:33.713 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150786.log 2023-07-06T21:43:33.714 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152542.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.152542.log.gz 2023-07-06T21:43:33.714 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144987.log 2023-07-06T21:43:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150786.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.150786.log.gz 2023-07-06T21:43:33.715 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145125.log 2023-07-06T21:43:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144987.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144987.log.gz 2023-07-06T21:43:33.716 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154349.log 2023-07-06T21:43:33.717 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145125.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145125.log.gz 2023-07-06T21:43:33.718 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155157.log 2023-07-06T21:43:33.718 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154349.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154349.log.gz 2023-07-06T21:43:33.719 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143416.log 2023-07-06T21:43:33.719 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155157.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155157.log.gz 2023-07-06T21:43:33.720 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145715.log 2023-07-06T21:43:33.721 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143416.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.143416.log.gz 2023-07-06T21:43:33.721 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136799.log 2023-07-06T21:43:33.722 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145715.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145715.log.gz 2023-07-06T21:43:33.722 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154257.log 2023-07-06T21:43:33.723 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136799.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136799.log.gz 2023-07-06T21:43:33.723 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145623.log 2023-07-06T21:43:33.724 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154257.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.154257.log.gz 2023-07-06T21:43:33.725 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146300.log 2023-07-06T21:43:33.725 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145623.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.145623.log.gz 2023-07-06T21:43:33.726 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149139.log 2023-07-06T21:43:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146300.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146300.log.gz 2023-07-06T21:43:33.727 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146832.log 2023-07-06T21:43:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.149139.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.149139.log.gz 2023-07-06T21:43:33.728 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.140809.log 2023-07-06T21:43:33.729 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146832.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146832.log.gz 2023-07-06T21:43:33.729 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156376.log 2023-07-06T21:43:33.731 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.2.140809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138389.log 2023-07-06T21:43:33.731 INFO:teuthology.orchestra.run.smithi084.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.2.140809.log.gz 2023-07-06T21:43:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156376.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.156376.log.gz 2023-07-06T21:43:33.732 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146104.log 2023-07-06T21:43:33.733 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138389.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.138389.log.gz 2023-07-06T21:43:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136615.log 2023-07-06T21:43:33.734 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146104.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.146104.log.gz 2023-07-06T21:43:33.735 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154395.log 2023-07-06T21:43:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136615.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136615.log.gz 2023-07-06T21:43:33.736 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151135.log 2023-07-06T21:43:33.737 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.154395.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154395.log.gz 2023-07-06T21:43:33.737 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146058.log 2023-07-06T21:43:33.738 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151135.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.151135.log.gz 2023-07-06T21:43:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143278.log 2023-07-06T21:43:33.739 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146058.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146058.log.gz 2023-07-06T21:43:33.740 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152868.log 2023-07-06T21:43:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143278.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143278.log.gz 2023-07-06T21:43:33.741 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150740.log 2023-07-06T21:43:33.742 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152868.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.152868.log.gz 2023-07-06T21:43:33.742 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141976.log 2023-07-06T21:43:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.150740.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150740.log.gz 2023-07-06T21:43:33.743 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138597.log 2023-07-06T21:43:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141976.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141976.log.gz 2023-07-06T21:43:33.744 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156984.log 2023-07-06T21:43:33.745 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138597.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138597.log.gz 2023-07-06T21:43:33.746 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143776.log 2023-07-06T21:43:33.747 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156984.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156984.log.gz 2023-07-06T21:43:33.747 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146440.log 2023-07-06T21:43:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143776.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143776.log.gz 2023-07-06T21:43:33.748 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139195.log 2023-07-06T21:43:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146440.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.146440.log.gz 2023-07-06T21:43:33.749 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144435.log 2023-07-06T21:43:33.750 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139195.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.139195.log.gz 2023-07-06T21:43:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158665.log 2023-07-06T21:43:33.751 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144435.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144435.log.gz 2023-07-06T21:43:33.752 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138372.log 2023-07-06T21:43:33.752 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158665.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.158665.log.gz 2023-07-06T21:43:33.753 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152588.log 2023-07-06T21:43:33.754 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138372.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.138372.log.gz 2023-07-06T21:43:33.754 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138643.log 2023-07-06T21:43:33.755 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152588.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152588.log.gz 2023-07-06T21:43:33.755 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139287.log 2023-07-06T21:43:33.756 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138643.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138643.log.gz 2023-07-06T21:43:33.756 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146394.log 2023-07-06T21:43:33.757 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.139287.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139287.log.gz 2023-07-06T21:43:33.758 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137309.log 2023-07-06T21:43:33.758 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146394.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146394.log.gz 2023-07-06T21:43:33.759 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144527.log 2023-07-06T21:43:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137309.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137309.log.gz 2023-07-06T21:43:33.760 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155636.log 2023-07-06T21:43:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144527.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.144527.log.gz 2023-07-06T21:43:33.761 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144711.log 2023-07-06T21:43:33.762 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.155636.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155636.log.gz 2023-07-06T21:43:33.762 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144389.log 2023-07-06T21:43:33.763 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144711.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144711.log.gz 2023-07-06T21:43:33.764 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152914.log 2023-07-06T21:43:33.764 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.144389.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144389.log.gz 2023-07-06T21:43:33.765 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142206.log 2023-07-06T21:43:33.766 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.152914.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152914.log.gz 2023-07-06T21:43:33.766 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158589.log 2023-07-06T21:43:33.767 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142206.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142206.log.gz 2023-07-06T21:43:33.767 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146716.log 2023-07-06T21:43:33.768 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158589.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158589.log.gz 2023-07-06T21:43:33.768 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157214.log 2023-07-06T21:43:33.769 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146716.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146716.log.gz 2023-07-06T21:43:33.769 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137653.log 2023-07-06T21:43:33.770 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157214.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157214.log.gz 2023-07-06T21:43:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138015.log 2023-07-06T21:43:33.771 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137653.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137653.log.gz 2023-07-06T21:43:33.772 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-07-06T21:43:33.773 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.138015.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.138015.log.gz 2023-07-06T21:43:33.773 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147757.log 2023-07-06T21:43:33.774 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143554.log 2023-07-06T21:43:33.775 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147757.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.147757.log.gz 2023-07-06T21:43:33.776 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156287.log 2023-07-06T21:43:33.777 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143554.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143554.log.gz 2023-07-06T21:43:33.780 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153962.log 2023-07-06T21:43:33.781 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.156287.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.156287.log.gz 2023-07-06T21:43:33.781 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147328.log 2023-07-06T21:43:33.781 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153962.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153962.log.gz 2023-07-06T21:43:33.781 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141654.log 2023-07-06T21:43:33.781 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.147328.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147328.log.gz 2023-07-06T21:43:33.782 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141838.log 2023-07-06T21:43:33.782 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141654.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141654.log.gz 2023-07-06T21:43:33.783 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157715.log 2023-07-06T21:43:33.783 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141838.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.141838.log.gz 2023-07-06T21:43:33.784 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137125.log 2023-07-06T21:43:33.784 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.157715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157715.log.gz 2023-07-06T21:43:33.785 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142506.log 2023-07-06T21:43:33.786 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137125.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137125.log.gz 2023-07-06T21:43:33.786 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153152.log 2023-07-06T21:43:33.787 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.142506.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142506.log.gz 2023-07-06T21:43:33.787 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145171.log 2023-07-06T21:43:33.788 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.153152.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.153152.log.gz 2023-07-06T21:43:33.789 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146670.log 2023-07-06T21:43:33.789 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.145171.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145171.log.gz 2023-07-06T21:43:33.790 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151503.log 2023-07-06T21:43:33.791 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.146670.log: gzip -5 --verbose -- 87.9%/var/log/ceph/ceph-mgr.y.log 2023-07-06T21:43:33.791 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/ceph-client.admin.146670.log.gz 2023-07-06T21:43:33.791 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.151503.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151503.log.gz 2023-07-06T21:43:33.792 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143462.log 2023-07-06T21:43:33.793 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158543.log 2023-07-06T21:43:33.794 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143462.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143462.log.gz 2023-07-06T21:43:33.795 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136347.log 2023-07-06T21:43:33.795 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.158543.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158543.log.gz 2023-07-06T21:43:33.796 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141102.log 2023-07-06T21:43:33.798 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.136347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143730.log 2023-07-06T21:43:33.798 INFO:teuthology.orchestra.run.smithi084.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136347.log.gz 2023-07-06T21:43:33.799 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.141102.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141102.log.gz 2023-07-06T21:43:33.799 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137652.log 2023-07-06T21:43:33.800 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143324.log 2023-07-06T21:43:33.800 INFO:teuthology.orchestra.run.smithi084.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143730.log.gz 2023-07-06T21:43:33.801 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.137652.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.137652.log.gz 2023-07-06T21:43:33.802 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/ceph-client.admin.143324.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143324.log.gz 2023-07-06T21:43:33.823 INFO:teuthology.orchestra.run.smithi084.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-07-06T21:43:33.843 INFO:teuthology.orchestra.run.smithi084.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-07-06T21:43:34.060 INFO:teuthology.orchestra.run.smithi084.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-07-06T21:43:34.064 INFO:teuthology.orchestra.run.smithi084.stderr: 2023-07-06T21:43:34.064 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m0.753s 2023-07-06T21:43:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:user 0m1.175s 2023-07-06T21:43:34.065 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m0.965s 2023-07-06T21:43:34.066 INFO:tasks.ceph:Archiving logs... 2023-07-06T21:43:34.066 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/remote/smithi084/log 2023-07-06T21:43:34.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T21:43:34.549 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/remote/smithi107/log 2023-07-06T21:43:34.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-06T21:43:34.750 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-06T21:43:34.778 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-06T21:43:34.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T21:43:34.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-06T21:43:34.905 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-06T21:43:34.914 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-06T21:43:34.915 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T21:43:34.919 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-06T21:43:34.955 INFO:teuthology.orchestra.run.smithi084.stderr:bash: ntpq: command not found 2023-07-06T21:43:34.960 INFO:teuthology.orchestra.run.smithi107.stderr:bash: ntpq: command not found 2023-07-06T21:43:34.963 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T21:43:34.963 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2023-07-06T21:43:34.963 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +1689us[+1683us] +/- 95ms 2023-07-06T21:43:34.964 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -1222us[-1229us] +/- 53ms 2023-07-06T21:43:34.964 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +379us[ +379us] +/- 71ms 2023-07-06T21:43:34.964 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:43:34.967 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-06T21:43:34.967 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2023-07-06T21:43:34.967 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +1696us[+1703us] +/- 95ms 2023-07-06T21:43:34.968 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -1194us[-1188us] +/- 53ms 2023-07-06T21:43:34.968 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +401us[ +408us] +/- 71ms 2023-07-06T21:43:34.968 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-07-06T21:43:34.969 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-06T21:43:34.983 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-06T21:43:34.985 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-06T21:43:35.009 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T21:43:35.183 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-06T21:43:35.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T21:43:35.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:43:35.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-06T21:43:35.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:43:35.496 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-06T21:43:35.506 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-06T21:43:35.546 INFO:teuthology.task.internal:Duration was 1179.210469 seconds 2023-07-06T21:43:35.547 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-06T21:43:35.579 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-06T21:43:35.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T21:43:35.584 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-06T21:43:35.696 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T21:43:35.699 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-06T21:43:36.161 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-06T21:43:36.161 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2023-07-06T21:43:36.162 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T21:43:36.220 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2023-07-06T21:43:36.221 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-06T21:43:36.279 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-06T21:43:36.280 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T21:43:36.285 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-06T21:43:36.368 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-06T21:43:36.368 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T21:43:36.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-06T21:43:36.501 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-06T21:43:36.512 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-06T21:43:36.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T21:43:36.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-06T21:43:36.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-06T21:43:36.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T21:43:36.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-06T21:43:36.725 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2023-07-06T21:43:36.729 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2023-07-06T21:43:36.764 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T21:43:36.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:43:36.805 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-06T21:43:36.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-06T21:43:36.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-06T21:43:36.857 INFO:teuthology.task.internal:Transferring archived files... 2023-07-06T21:43:36.858 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/remote/smithi084 2023-07-06T21:43:36.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T21:43:36.938 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242/remote/smithi107 2023-07-06T21:43:36.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-06T21:43:37.018 INFO:teuthology.task.internal:Removing archive directory... 2023-07-06T21:43:37.018 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T21:43:37.023 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-06T21:43:37.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-06T21:43:37.077 INFO:teuthology.task.internal:Not uploading archives. 2023-07-06T21:43:37.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-06T21:43:37.139 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-06T21:43:37.139 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T21:43:37.144 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-06T21:43:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 21:43 /home/ubuntu/cephtest 2023-07-06T21:43:37.186 INFO:teuthology.orchestra.run.smithi107.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 6 21:43 /home/ubuntu/cephtest 2023-07-06T21:43:37.189 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-06T21:43:37.200 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-06T21:43:37.255 INFO:teuthology.nuke:Checking targets against current locks 2023-07-06T21:43:37.289 DEBUG:teuthology.nuke:shortname: smithi084 2023-07-06T21:43:37.290 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T21:43:37.318 DEBUG:teuthology.nuke:shortname: smithi107 2023-07-06T21:43:37.318 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-06T21:43:37.325 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 21:14:29.566014', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-06T21:43:37.341 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-06_19:46:43-fs-wip-rishabh-CephManager-in-CephFSTestCase-testing-default-smithi/7328242', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-06 21:14:29.568095', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-06T21:43:37.382 INFO:teuthology.orchestra.console:Power off smithi084 2023-07-06T21:43:37.383 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T21:43:37.406 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:37.427 INFO:teuthology.orchestra.console:Power off smithi107 2023-07-06T21:43:37.428 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-06T21:43:37.451 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:41.441 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:41.476 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:45.556 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:45.592 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:45.805 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2023-07-06T21:43:49.671 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi084.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-06T21:43:49.885 INFO:teuthology.orchestra.console:Power off for smithi084 completed 2023-07-06T21:43:49.991 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/pool-perm} duration: 1179.2104694843292 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default mon.a-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd mon.b-kernel-sha1: 84139c18d56f518615369e8310ae4ecf07a36dcd owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=02002e561b714793af73106e6a1af032 status: fail success: false 2023-07-06T21:43:49.992 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-06T21:43:50.068 INFO:teuthology.run:FAIL